Friday, March 21, 2014

Stock Me Baby - Stock Analysis and Notification

In today's trading business it is very essential to keep traders aware of the latest happenings related to their investments. It is very essential for one to know when to invest more or sell his/her securities at the right moment for more gains and less losses. StockMeBaby is an extendable Java project focused on giving insightful alerts by scraping stock related information from the web and sending them as an alert to subscribers to help them manage their securities. This tool can be used for free by developers and subscribers.
How to extend StockMeBaby ?
. Implement the com.stockmebaby.scraper.Scrapable interface.
. An example can be found in this class com.stockmebaby.scraper.StreeInsider.
. The example shows how to scrape analyst ratings from
. This website is a popular source to keep track of stock ratings.
. Notification is only with a GMail ID.
. HtmlUnit is an easy to use API for scraping.
. The configurations for the project can be seen in at the root of the project.

Thursday, March 13, 2014

Groovy config slurper and java perm gen memory pool

ConfigSlurper is a utility class within Groovy for writing properties file like scripts for performing configuration. Unlike regular Java properties files ConfigSlurper scripts support native Java types and are structured like a tree facilitating namespace reuse. Since a property file is a script a new class is created when the configuration file is read using the ConfigSlurper. 

PermGen is a memory pool used by the JVM to store loaded class files. Following is a piece of code that loads the configuration file using Groovy's ConfigSluper 10K times. I have also taken a screen shot of JConsole showing us the number of loaded class over time on a graph. I saw this code end with an OutOfMemoryException because at some point there was no space in PermGen memory pool for more classes to be loaded.

Sunday, August 4, 2013

Grails compile test files from command prompt

When you type in "grails compile" at the command prompt Grails compiles only the source files and does not compile the test files. It sometimes becomes necessary to compile the test files and make it a part of your project's build process because every code check in does not need to run the tests especially if the change in code is very trivial and the test run time is too long. This is very useful when you have Grails and Java projects in co-existence. The following Gant script when included under GRAILS_HOME/scripts folder will enable us to compile integration and unit test files from the command prompt by typing "grails compile-tests". 

Gant script download: CompileTests.groovy

Tuesday, July 23, 2013

How to log all MySql queries into mysql.slow_log table ?

Sometimes it becomes necessary that you see the history of queries that your MySql server ran. This is useful while debugging your application. In order to make MySql log all queries that it runs from multiple clients into mysql.slow_log table you can run the following SQL statements:

// Enables the slow query log feature.
SET GLOBAL slow_query_log = 1;

// This tells MySql to log the queries to a table (you can also log queries to a file).
SET GLOBAL log_output = 'TABLE';

// This logs queries which take more than 0 units of time to run.
SET GLOBAL long_query_time = 0;

// This logs queries which examines 0 or more rows.
SET GLOBAL min_examined_row_limit = 0;

// This logs queries which do not use indexes (the name says it all).
SET GLOBAL log_queries_not_using_indexes = 1;

// You can now see the queries being logged.
SELECT * FROM mysql.slow_log;

// Delete the logs using this statement.
TRUNCATE mysql.slow_log;

Tuesday, April 30, 2013

MySQL InnoDB Table Optimization Performance

The InnoDB that comes with MySQL as of 5.6.11 has a serious flaw when it comes to utilizing the Fast Index Creation offered by MySQL when running optimize table queries. This is a verified bug which was reported a long time ago and the link to the bug is here. The reason why InnoDB still doesn't have this feature is because I guess Oracle thinks that this is more a feature than a bug. Percona Server has resolved this issue in their XtraDB storage engine but it comes at a cost :)

So i have a table with one primary key and one secondary key and the definition of the table is below. The table is populated with 240753 records. The information schema tells that the data length of the table is 22593536 and the innodb_buffer_pool_size is 5242880. So the table size is approximately 4 times the buffer size. 

  id int(10) unsigned NOT NULL AUTO_INCREMENT,
  md5hash char(64) DEFAULT NULL,
  KEY mykey (md5hash)

Tuesday, March 26, 2013

Self referencing an EJB to start a new transaction with Container Managed Transactions

Let us take an example of a Stateless EJB that has transaction attribute as REQUIRED. This means that if a client invokes the bean's method while the client is associated with a transaction context then the container invokes the bean's method in the client's transaction context. If the client invokes the bean's method while the client is not associated with a transaction context then the container starts a new transaction before it invokes the beans's method.

@TransactionAttribute( value = TransactionAttributeType.REQUIRED )
class EJB1 implements EJB1Local{
    public void method1( ){ }

   @TransactionAttribute( value = TransactionAttributeType.REQUIRES_NEW )
    public void method2( ){ }


Monday, March 25, 2013

Overriding default transaction control in JBoss

As of JBoss 5.0.1.GA the transaction time out attribute can be set explicitly in the configuration file named transaction-jboss-beans.xml. In JBoss CMT ( Container Managed Transactions ) if a transaction takes more time than the time specified in the configuration file then an Exception is thrown.

Sometimes purging a database table and optimizing it might be in the same transaction. Optimizing a table need not be a transaction whereas purging a table definitely has to be a part of the transaction as it leads to inconsistencies if not committed. Optimizing a table might take a lot of time if the DB is large. It is definitely not a good idea to combine purging and optimizing in one transaction because this might lead to a transaction timeout.