Merging MCSEM.com

MCSEM.com was a website I had maintained to provide my marine CSEM modelling software to the public. That project has now come to a close and I would like to focus my endeavors on topics not so specific to a single geophysical method. So instead of maintaining two website I am in the process of merging the content from MCSEM.com to Digital Earth Lab. There will be some duplication in the meantime and there the site may be a little messy, so bare with me.

So if you would like to see what MCSEM.com was about check out the following sections,

Introduction to Marine CSEM

or check out the previous entries,

By |June 21st, 2014|Uncategorized|0 Comments

Reading Large ASCII Datasets

Ever tried reading large geophysical ASCII data sets with a standard text editor only to get angry when it hangs, crashes or runs out of memory? Chances are you don't even want to edit the document but rather want to read the first one or two lines in the header. There are several solutions to solve this problem. First of all you can load the data into a software package which imports the data directly into a database (i.e., Oasis Montaj), but that requires you to load hundreds of megabytes or even  gigabytes of data, wasting minutes in the process. Secondly, if you are 'fortunate' enough to use a Unix based OS, you can use the "less" command, great for Linux users but pretty much worthless for Windows users.

If you want to try another solution out see below.

The source code is downloadable and is licensed under WTFPL. Distribute, copy, modify, sell, integrate, delete, make erroneous.... I really don't care what you do with it. The below source code is only a sample and does not represent the complete source code.
The premise
The header in a large ASCII text file can be read by storing one line at a time in memory. This avoids memory issues and consumes mere kilobytes of memory rather than trying to import gigabytes at a time.
The Solution
I have written some code using the JDK 8 developers release SDK containing the latest JavaFX libraries. This can be run under the Windows OS. The aptly named perpetual ASCII file viewer reads a file in blocks. Using Java this is accomplished by using the lines,

BufferedReader in;
public final TextArea t = new TextArea(); //Text area displaying the read text
private int preload, bufferSize; //preload 'preload' lines and load [...]

By |April 9th, 2014|News, Uncategorized|0 Comments