performance - "Fastest" hash function implemented in Java, comparing part of file -

-

i need compare 2 different files of instance "file" in java , want fast hash function.

idea: - hashing 20 first lines in file 1 - hashing 20 first lines in file 2 - compare 2 hashes , return true if equal.

i want use "fastest" hash function ever been implemented in java. 1 choose?

if want speed, not hash! not cryptographic hash md5. these hashes designed impossible reverse, not fast calculate. should use checksum - see java.util.zip.checksum , 2 concrete implementations. adler32 extremely fast compute.

any method based on checksums or hashes vulnerable collisions, can minimise risk using 2 different methods in way rsync does.

the algorithm basically:

  • check file sizes equal
  • break files chunks of size n bytes
  • compute checksum on each pair of matching blocks , compare. differences prove files not same.

this allows detection of difference. can improve computing 2 checksums @ once different algorithms, or different block sizes.

more bits in result mean less chance of collision, go on 64 bits outside java (and computer's cpu) can handle natively , hence slow, fnv-1024 less give false negative slower.

if speed, use adler32 , accept difference not detected. rare. checksums these used ensure internet can spot transmission errors, , how wrong data turning up?

it accuracy really, have compare every byte. nothing else work.

if can compromise between speed , accuracy, there wealth of options out there.


Comments

Post a Comment

Popular posts from this blog

php - What is the difference between $_SERVER['PATH_INFO'] and $_SERVER['ORIG_PATH_INFO']? -

-
what’s difference between $_server['path_info'] , $_server['orig_path_info'] ? how use them? when run print_r($_server) , path_info , orig_path_info not present in array. why not? how can enable them? i have read php manual on them, still don’t understand them. the path_info variable present if invoke php script this: http://www.example.com/phpinfo.php/hello_there it's /hello_there part after .php script. if don't invoke url that, there won't $_server["path_info"] environment variable. the porig_ prefix uncommon. path_info standard cgi-environment variable, , should never prefixed. did read that? (there issues around php3/php4 if invoked php interpreter via cgi-bin/ - hardly has such setups today.) for reference: http://www.ietf.org/rfc/rfc3875
Read more

fortran - Function return type mismatch -

-
i'm attempting recode old c++ program in fortran make use of lapack (i'm aware c++ have lapack++, i'm having lot of trouble installing it, gave up). i didn't have problems compilation, when had variables declared real . when started coding section of program required lapack, found parameters passed dsyev need double precision . tried change double precision (including changing hard coded numbers double precision counterparts, i.e. 0.0 -> 0.0d0) when try compile, following error of functions , subroutines i've written: error: return type mismatch of function <function> @ (1) (real(4)/real(8)) i'm not sure coming from, in program has been changed double precision. for example, i've declared following: double precision :: alpha(3),d(3),zeta1,zeta2 double precision :: a1(3),a2(3),d1(3),d2(3) double precision :: pi pi = 3.14159265359d0 alpha = (/0.109818d0, 0.405771d0, 2.22766d0/) d = (/0.444635d0, 0.535328d0, 0.154329d0 /) 10 i=1,3 ...
Read more

queue - mq_receive: message too long -

-
i implementing communication between 2 processes using queue. problem when call function mq_receive, error: message long. i have done following: struct mq_attr attr; long size = attr.mq_msgsize; .... // initializing queue "/gateway" int rc = mq_receive(gateway, buffer, size, &prio); if print size value, size=1, while when print same size program (got same mechanism), not long integer ( -1217186280 )... how can solve error?....so while size = 1, believe it's right "message long" why 1? p.s. have tried put : int rc = mq_receive(gateway, buffer, sizeof(buffer), &prio); but no result. it seems need read docs more carefully. when call mq_receive should pass size of destination buffer. this size must greater mq_msgsize attribute of queue . in addition, seems have error in queue attributes initialisation makes proper mq_receive call impossible. here standard message queue session: fill mq_attr struct ( doc ): struct mq_a...
Read more