AnsweredAssumed Answered

How to set FsPermission efficiently?

Question asked by matroyd on Jul 6, 2012
Latest reply on Jul 10, 2012 by matroyd
We are currently writing to MapR using HDFS API, however each write (out of several hundreds-thousands) involves two calls (create and setPermission).

Is there a better efficient way to do this in one shot/roundtrip?

Is there a way to set the default file permission and ownership at Volume level?

current code snippet:
              
            ByteArrayOutputStream bos = null;
      ObjectOutput out = null;
      FSDataOutputStream ostr = null;
      FileSystem fs = null;
      Configuration conf = new Configuration();
      try{
       fs = FileSystem.get(conf);
       Path wfilepath = new Path(filePath);
       try {
        bos = new ByteArrayOutputStream();
        out = new ObjectOutputStream(bos);
        out.writeObject(data);
        out.flush();
        byte[] rwBytes = bos.toByteArray();
        ostr = fs.create(wfilepath, // path
          true, // overwrite
          64 * 1024 * 1024, // buffersize
          (short) 1, // replication
          (long) (128 * 1024 * 1024) // chunksize
          );
        ostr.write(rwBytes);
        ostr.flush();
       } catch (Exception e) {
        e.printStackTrace();
       } finally {
        if (out != null) {
         try {
          out.close();
         } catch (IOException e) {
          e.printStackTrace();
         }
        }
        if (bos != null) {
         try {
          bos.close();
         } catch (IOException e) {
          e.printStackTrace();
         }
        }
        if (ostr != null) {
         try {
          ostr.close();
         } catch (IOException e) {
          e.printStackTrace();
         }
        }
       }
       fs.setPermission(wfilepath, fsp);
      }
      catch(Exception e){
       e.printStackTrace();
      }finally{
       if(fs!=null){
        try {
         fs.close();
        } catch (IOException e) {
         e.printStackTrace();
        }
       }
      }

Outcomes