performance - Slow array operations -

-

i'm quite new matlab programmer, might easy one.. :)

i'm trying generate script, able read number of xyz-files, in order, array, , arrange them in array according x , y coordinates given in file.. attempt use load files array, , after that, read through array and, explained, use x , y coordinate locations in new array..

i've tried presetting array size, , i'm subtracting value both x , y minimize size of array (fullarray)

%# script extraction of xyz-data dsm/dtm xyz files  %# define folders , filter dsmfolder='/share/cfdwork/site/ofsites/mabh/dsm/*.xyz'; dtmfolder='/share/cfdwork/site/ofsites/mabh/dtm/*.xyz';  %# define minimumvalues, reduce arrays.. please leave slack, %#  reduction-algorithm.. borderx=100000; bordery=210000;  %% expected array-size expsizex=20000; expsizey=20000;    %# program starts.. please not edit below line! files=ls(dsmfolder);  clear fullarray fullarray=zeros(expsizex,expsizey);  minx=999999999; miny=999999999; maxx=0; maxy=0;  disp('reading dsm files'); [thisfile,remaining]=strtok(files); while (~isempty(thisfile))     disp(['reading: ' thisfile]);     clear fromfile;     fromfile=load(thisfile);     k=1:size(fromfile,1)         tic         fullarray(fromfile(k,1)-borderx,fromfile(k,2)-bordery)=fromfile(k,3);         disp([k size(fromfile,1)]);         if (fromfile(k,1)<minx)             minx=fromfile(k,1);         end         if (fromfile(k,2)<miny)             miny=fromfile(k,2);         end         if (fromfile(k,1)>maxx)             maxx=fromfile(k,1);         end         if (fromfile(k,2)>maxy)             maxy=fromfile(k,2);         end         toc     end     [thisfile,remaining]=strtok(remaining); end

as can seen, i've added tic-toc, , time 3.36secs 1 operation!

any suggestion on, why slow, , how improve speed.. need order 2x6,000,000 lines, , can't bothered wait 466 days.. :d

best regards mark

have considered using sparse matrix?

a sparse matrix in matlab defined list of values , location in matrix - incidentally matches input file perfectly.

while representation meant matrices sparse, (i.e. of values zeros), appears in case faster load matrix using sparse function if not sparse.

since data organised in such way (location of every data point) guess is sparse anyway.

the function create sparse matrix takes location columns instead of loop code (this segment replaces whole loop): 

minx = min(fromfile(:,1); maxx = max(fromfile(:,1); miny = min(fromfile(:,2); miny = max(fromfile(:,2); s = sparse(fromfile(:,1) - borderx, fromfile(:,2) - bordery, fromfile(:,3));

note other change i've made calculating minimum / maximum values directly matrix - faster going on loop, operating on vectors , matrices unleashes true power of matlab :)

you can perform sorts of operations on sparse matrix, if want convert regular matrix can use matlab full function.

more information here , there.


Comments

Post a Comment

Popular posts from this blog

php - What is the difference between $_SERVER['PATH_INFO'] and $_SERVER['ORIG_PATH_INFO']? -

-
what’s difference between $_server['path_info'] , $_server['orig_path_info'] ? how use them? when run print_r($_server) , path_info , orig_path_info not present in array. why not? how can enable them? i have read php manual on them, still don’t understand them. the path_info variable present if invoke php script this: http://www.example.com/phpinfo.php/hello_there it's /hello_there part after .php script. if don't invoke url that, there won't $_server["path_info"] environment variable. the porig_ prefix uncommon. path_info standard cgi-environment variable, , should never prefixed. did read that? (there issues around php3/php4 if invoked php interpreter via cgi-bin/ - hardly has such setups today.) for reference: http://www.ietf.org/rfc/rfc3875
Read more

fortran - Function return type mismatch -

-
i'm attempting recode old c++ program in fortran make use of lapack (i'm aware c++ have lapack++, i'm having lot of trouble installing it, gave up). i didn't have problems compilation, when had variables declared real . when started coding section of program required lapack, found parameters passed dsyev need double precision . tried change double precision (including changing hard coded numbers double precision counterparts, i.e. 0.0 -> 0.0d0) when try compile, following error of functions , subroutines i've written: error: return type mismatch of function <function> @ (1) (real(4)/real(8)) i'm not sure coming from, in program has been changed double precision. for example, i've declared following: double precision :: alpha(3),d(3),zeta1,zeta2 double precision :: a1(3),a2(3),d1(3),d2(3) double precision :: pi pi = 3.14159265359d0 alpha = (/0.109818d0, 0.405771d0, 2.22766d0/) d = (/0.444635d0, 0.535328d0, 0.154329d0 /) 10 i=1,3 ...
Read more

queue - mq_receive: message too long -

-
i implementing communication between 2 processes using queue. problem when call function mq_receive, error: message long. i have done following: struct mq_attr attr; long size = attr.mq_msgsize; .... // initializing queue "/gateway" int rc = mq_receive(gateway, buffer, size, &prio); if print size value, size=1, while when print same size program (got same mechanism), not long integer ( -1217186280 )... how can solve error?....so while size = 1, believe it's right "message long" why 1? p.s. have tried put : int rc = mq_receive(gateway, buffer, sizeof(buffer), &prio); but no result. it seems need read docs more carefully. when call mq_receive should pass size of destination buffer. this size must greater mq_msgsize attribute of queue . in addition, seems have error in queue attributes initialisation makes proper mq_receive call impossible. here standard message queue session: fill mq_attr struct ( doc ): struct mq_a...
Read more