java - How to efficiently convert character encoding in Channel? -

-

i need accept streams different encodings , transcode them single predefined encoding (e.g. utf-8). know how (inputstream)reader / (outputstream)writer combo , array buffer, time i'm dealing bytechannel's. naturally, looking charsetdecoder / charsetencoding solution, best came is:

public static void copy(readablebytechannel rbc, charset in,          writablebytechannel wbc, charset out) throws ioexception {     bytebuffer b1 = bytebuffer.allocatedirect(buffer_size);     charbuffer cb = charbuffer.allocate(buffer_size);     bytebuffer b2 = bytebuffer.allocatedirect(buffer_size);      charsetdecoder decoder = in.newdecoder();     decoder.onmalformedinput(codingerroraction.replace);     charsetencoder encoder = out.newencoder();     encoder.onunmappablecharacter(codingerroraction.replace);      while( rbc.read(b1)!=-1 ){         b1.flip();         decoder.decode(b1, cb, false);         cb.flip();         encoder.encode(cb, b2, false);         b2.flip();         wbc.write(b2);         b2.compact();         cb.compact();         b1.compact();     }     b1.flip();     while (b1.hasremaining()){         decoder.decode(b1, cb, true);         cb.flip();         encoder.encode(cb, b2, false);         b2.flip();         wbc.write(b2);         b2.compact();         cb.compact();     }     decoder.decode(b1, cb, true);     decoder.flush(cb);     cb.flip();     while (cb.hasremaining()){         encoder.encode(cb, b2, true);         b2.flip();         wbc.write(b2);         b2.compact();     }     encoder.encode(cb, b2, true);     encoder.flush(b2);     b2.flip();     while (b2.hasremaining()){         wbc.write(b2);     } }

since method "workhorse" in project, must absolutely sure finish no matter combination of buffer_size, encodings , blocking device output given.

my questions are:

  • is there better way of buffer draining instead of these cascade of while loops?
  • is ok ignore encode() / decode() results (for overflows , underflows)?

of course, alternative idea welcomed. :)

to improve performance of above code:

  1. cache byte/char buffers in thread local or fields. allocating chunks of memory expensive.
  2. direct byte buffers performers io bad performers encoding/decoding has optimized implementation heap buffers. might better performance copying to/from heap byte buffers decode/encode operations.
  3. you can skip encode/decode when charset same.
  4. minimize calls compact.
  5. you seem have redundant decode/encode operations after buffer have nothing remaining.
  6. byte buffer size should 4 time char buffer size, chars can 1-4 bytes. allocating byte buffers multiples of page size(usually 4k) can io performance.

most importantly, write benchmark realistic data , use means measure performance improvements. if don't measure you'll never know worked.


Comments

Post a Comment

Popular posts from this blog

php - What is the difference between $_SERVER['PATH_INFO'] and $_SERVER['ORIG_PATH_INFO']? -

-
what’s difference between $_server['path_info'] , $_server['orig_path_info'] ? how use them? when run print_r($_server) , path_info , orig_path_info not present in array. why not? how can enable them? i have read php manual on them, still don’t understand them. the path_info variable present if invoke php script this: http://www.example.com/phpinfo.php/hello_there it's /hello_there part after .php script. if don't invoke url that, there won't $_server["path_info"] environment variable. the porig_ prefix uncommon. path_info standard cgi-environment variable, , should never prefixed. did read that? (there issues around php3/php4 if invoked php interpreter via cgi-bin/ - hardly has such setups today.) for reference: http://www.ietf.org/rfc/rfc3875
Read more

fortran - Function return type mismatch -

-
i'm attempting recode old c++ program in fortran make use of lapack (i'm aware c++ have lapack++, i'm having lot of trouble installing it, gave up). i didn't have problems compilation, when had variables declared real . when started coding section of program required lapack, found parameters passed dsyev need double precision . tried change double precision (including changing hard coded numbers double precision counterparts, i.e. 0.0 -> 0.0d0) when try compile, following error of functions , subroutines i've written: error: return type mismatch of function <function> @ (1) (real(4)/real(8)) i'm not sure coming from, in program has been changed double precision. for example, i've declared following: double precision :: alpha(3),d(3),zeta1,zeta2 double precision :: a1(3),a2(3),d1(3),d2(3) double precision :: pi pi = 3.14159265359d0 alpha = (/0.109818d0, 0.405771d0, 2.22766d0/) d = (/0.444635d0, 0.535328d0, 0.154329d0 /) 10 i=1,3 ...
Read more

queue - mq_receive: message too long -

-
i implementing communication between 2 processes using queue. problem when call function mq_receive, error: message long. i have done following: struct mq_attr attr; long size = attr.mq_msgsize; .... // initializing queue "/gateway" int rc = mq_receive(gateway, buffer, size, &prio); if print size value, size=1, while when print same size program (got same mechanism), not long integer ( -1217186280 )... how can solve error?....so while size = 1, believe it's right "message long" why 1? p.s. have tried put : int rc = mq_receive(gateway, buffer, sizeof(buffer), &prio); but no result. it seems need read docs more carefully. when call mq_receive should pass size of destination buffer. this size must greater mq_msgsize attribute of queue . in addition, seems have error in queue attributes initialisation makes proper mq_receive call impossible. here standard message queue session: fill mq_attr struct ( doc ): struct mq_a...
Read more