Unravelling the black magic of bureaucracy

Arthur C. Clarke once famously wrote “Any sufficiently advanced technology is indistinguishable from magic”. In the same vein, many people feel that any sufficiently established bureaucracy is like a black magic, sorcery even. Certainly, it often takes skills out of this world to follow the logic of modern tax return instructions. Bureaucracy often has its place and reason. Laws protect exploitable minorities; procedures serve to avoid known problems; cross-referencing forms are filled in triplicate to allow for audit and protection against falsification.

Reducing disk thrashing of OpenNLP/MaxEnt parser – with one line code change

When OpenNLP toolkit uses MaxEnt parser, it has to read in about 25 MBytes of model files. The model reader uses basic unbuffered FileReader. The result is the excessive number of system calls (and disk access calls) during the parser startup. The fix is extremely simple: In maxent-2.4.0/src/java/opennlp/maxent/io/ObjectGISModelReader.java, replace new FileInputStream(f) with new BufferedInputStream(new FileInputStream(f), 1000000) Recompile maxent library Deploy new version of maxent-2.4.0.jar into OpenNLP’s lib directory The comparison is striking (the numbers are File access system calls):

Getting OpenNLP parser to work

I was not able to get OpenNLP parser to work. There were no samples to play with, no command line tools to run. And I don’t even want to talk about documentation. That’s because there was not any. There was an attempt at lame joke (at least that’s the only sense I can make of what.html file), but no actual documentation. Finally, I pinged my research colleague who did get the toolkit working (thanks Scott).

Running Bikel’s parser programmatically

Bikel’s statistical parser is designed to be run from the command line. I need to run it from my own code. The following wrapper seems to do the trick on windows (with your own values for|parserdir| : `Bikel’s statistical parser is designed to be run from the command line. I need to run it from my own code. The following wrapper seems to do the trick on windows (with your own values for|parserdir| :

Duplicating -tagSeparator effect when using Stanford Parser programmatically

I have been using Stanford NLP Parser from command line with -tagSeparator flag to supply it with partially tagged input. As the parser seems to be really bad with date expressions and complex name entities, I need this functionality. Now, I need to wrap-up the parser in my own code to add input/output batching and I discover that this option is not accepted when constructing parser from the code. Despite javadoc saying that LexicalizedParser.