Information Technology Tips

高性能なスペルミス修正アルゴリズムを How to Write a Spelling Corrector で見つけたので紹介します。 リンク先では、理論的背景とコードも説明されていますので、参考になるかと思います。 ここ のJavaバージョンのコードを私なりに書き直してみました。 package com.dukesoftware.spellcorrector; import java.io.IOException; import java.util.ArrayList; import java.util.Collections; import java.util.HashMap; import java.util.List; import java.util.Map; public class SpellCorrector { public static void main(String args[]) throws IOException { Map<String, Integer> nWords = new HashMap<String, Integer>() {{ put("spell", 1); }}; SpellCorrector spellCorrector = new SpellCorrector(nWords); System.out.println(spellCorrector.correct("superl")); } private final Map<String, Integer> nWords; private final String[] a_to_z; public SpellCorrector(Map<String, Integer> nWords) throws IOException { this.nWords = nWords; this.a_to_z = createAtoZStringArray(); } private Str...

Improve Performance for MySQL CALC_FOUND_ROWS with LIMIT OFFSET Push State Push state is html5 technology that records browsing history in javascript without actual page refresh. Push state is useful for recording Ajax request, which cannot record the browsing history normally, and telling the corresponding url for the Ajax request to browser. The minimum push state code is below: // push the browser state if (typeof(history.replaceState) !== "undefined") { history.pushState({prop1: "a", prop2:"b"}, null, "/url/?param1=param"); } // when browser back button is pressed, pop state event is fired // pop state handler is registered in the folloing code window.onpopstate = function (event) { if (event.state == null) { return; } // do something nice ;) // using "event.state.prop1" for example }; The usage of push state is quite simple. One thing you should...

SQL_CALC_FOUND_ROWS Performance Issue If you are working on MySQL (<=5.5) for a long time, you have an experience with performance issue on SQL_CALC_FOUND_ROWS with LIMIT OFFSET. The core cause is MySQL generates full result set as a temporary table even if user only requested first 20 rows in the query. This kind of situation often happens when you implement paging functionality. Please see following example. SELECT SQL_CALC_FOUND_ROWS tbl_a.a_id, tbl_b.*, tbl_c.*, FROM tbale_a AS tbl_a INNER JOIN table_b AS tbl_b ON tbl_b.a_id = tbl_a.a_id INNER JOIN table_c AS tbl_c ON tbl_c.b_id = tbl_b.b_id LEFT JOIN table_d AS tbl_d ON tbl_d.a_id = tbl_a.a_id WHERE tbl_b.field1 = "ABC" LIMIT 100, 20; In this example, a temporary table which has full result set is generated. If you configure SQL_CACHE, the second time query might be quite faster. However it is not a good choice because the query performance depends on status of tables, and not stable. Assume if tbl_b is fr...

If you want to test a class which depends on Google App Engine infrastructure, such as data storing functionality with "PersistenceManagerFactory", you should set up LocalServiceTestHelper before testing. I don't tell you the details very much, but I will show you the minimum test setup in the code below. This code was enough for me to test my data access object functionality. package com.dukesoftware.gaej.test; import org.junit.AfterClass; import org.junit.BeforeClass; import org.junit.Test; import com.google.appengine.tools.development.testing.LocalDatastoreServiceTestConfig; import com.google.appengine.tools.development.testing.LocalServiceTestHelper; public class GoogleAppEngineTest { private static final LocalServiceTestHelper helper = new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig()); @BeforeClass public static void setUp() { helper.setUp(); } @AfterClass public static void tearDown() { he...

When to Use Google App Engine Task Queue? You should use task queue when you try to do something taking time such as storing or updating bunch of data to datastore. In Google App Engine, a http request which isn't returned by timeout should be simply failed. The timeout configured on Google App Engine is 60 seconds. Please see https://developers.google.com/appengine/articles/deadlineexceedederrors . So in order to response back to the client quickly, you should use Task Queue for tasks which takes long time. Push Tasks to Task Queue I have read an official document of Task Queue and tried to use it. What frustrates me is the document only explains how to create a task with parameters, push the task to the task queue and pass to the worker servlet. i.e. actual heavy part is treated in the worker servlet. Of course it is fine, but I felt it was a bit indirect way to push tasks to queue. What I desired to do is creating task object and pushing it to the queue in one single...

If you want to get all static fields in a class, use code snippet below. The key part is "Modifier.isStatic", "getDeclaredFields" methods package com.dukesoftware.reflection; import java.lang.reflect.Field; import java.lang.reflect.Modifier; import java.util.Arrays; import java.util.List; import java.util.stream.Collectors; public class ReflectionUtils { public static List<Field> getStaticFields(Class<?> clazz) { Field[] declaredFields = clazz.getDeclaredFields(); return Arrays.stream(declaredFields) .filter(field -> Modifier.isStatic(field.getModifiers())) .collect(Collectors.toList()) ; } }

Currently I am working on optimizing image size. In this post, I will demonstrate the C# code for creating indexed png image. The code might not be perfect yet, but hope this code will give you some hints ;) A few things I am struggling with while I am writing the example code: Palette: you should take ColorPalette from Bitmap, modify the ColorPalette object and set buck it to the Bitmap object. BitmapData.Stride is often bigger than BitmapData.Width. You should be careful to check both values difference when you manipulating the image array index. The example code uses classes in System.Drawing namespace. But looks there is another option to manipulate image in C# .Net - System.Windows.Media namespace. C# Source Code for Creating Indexed PNG The following implementation of CreateIndexedPng method is able to take only non-indexed image like Format24bppRgb or Format32bppArgb. (I might add other image formats in future.) If the source image has only 2 colors, we can use For...

Save Excel Sheet as UTF-8 CSV file The biggest problem of Excel is that Excel does not support saving file as UTF-8 CSV format. I have googled and tried to find the solution for saving excel as UTF-8 CSV but all of them requires 2 steps - 1) saving Excel file as Unicode csv 2) Open it by text editor and save it again as UTF-8 CSV. If you are a software developer, it is easy to write Java or C# program to manipulate Excel. But it means people (non developer users) need to install or launch the application. So I have wrriten Excel VBA addin for saving Excel sheet as UTF-8 CSV file so that we can save UTF-8 excel files only using Excel. Code for saving UTF-8 CSV Okie!, here is my solution! Sme notes for the code: The code saves an active sheet to UTF-8 CSV file in temp folder. The error handring part is not robust. I have only wrriten minimum error handling. You might need to add your custome error handring. If you are okay to add BOM on the file, please remove the corresp...

Reflective Getter and Setter Method In this post, I will show you Java code for getting getter and setter method from given object. The code is below: public final static Method getSetterMethod(Object o, String propertyName, Class<?> paramterType) throws SecurityException, NoSuchMethodException{ return o.getClass().getMethod("set"+toUpperFirstChar(propertyName), paramterType); } public final static Method getGetterMethod(Object o, String propertyName) throws SecurityException, NoSuchMethodException{ return o.getClass().getMethod("get"+toUpperFirstChar(propertyName)); } public final static String toUpperFirstChar(String str){ if(str.isEmpty()) return ""; return str.substring(0, 1).toUpperCase()+str.substring(1, str.length()); } Where to Use Reflective Getter and Setter? You may have a question - "Where should we use reflective getter and setter method?" If you already know where to use them, you can skip the fo...

Code using System; using System.Collections.Generic; using System.Linq; namespace Utility { public static class ArrayUtils { public static T[,] To2DimensionalArray (this T[] source, int block){ var ret = new T[block, source.Length/block]; for (int i = 0, offset = 0, len = ret.Length; offset Example Usage int[] array1 = { 1, 2, 3, 4, 5, 6, 7, 8, 9}; int[,] array2 = array1.To2DimensionalArray(3); /* array2 will be... { { 1, 2, 3 }, { 4, 5, 6 }, { 7, 8, 9 } } */

Code using System; using System.Collections.Generic; using System.Linq; namespace Utility { public static class ArrayUtils { public static T[][] To2DimensionalJaggedArray<T>(this T[] source, int block) { var res = new T[block][]; int size = source.Length / block; for (int i = 0; i < block; i++) { res[i] = new T[size]; Array.Copy(source, i*size, res, 0, size); // advantage of jagged array is that you can use Array.copy method etc. } return res; } } } Example Usage int[] array1 = { 1, 2, 3, 4, 5, 6, 7, 8, 9}; int[][] array2 = array1.To2DimensionalJaggedArray(3); /* array2 will be int[3][] and each elements will be... [0] = { 1, 2, 3 }, [1] = { 4, 5, 6 }, [2] = { 7, 8, 9 } */

In this post, I will explain the simplest setup to use Amazon Product API in Java. I know Amazon provides soap interface and we can automatically create Soap code for accessing Amazon Product API. But sometimes, it is too much for light users. So I will show you the simplest way to use Amazon Product API using SignedRequestsHelper. Hope this post helps your application development ;) Preparation Create an AWS Account Get AWS Key, secret key and associate tag: I think you can create them from http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html Get SignedRequestsHelper: You can download from https://code.google.com/p/amazon-product-advertising-api-sample/source/browse/src/com/amazon/advertising/api/sample/SignedRequestsHelper.java Java Code Example Congratulation! After you finish above preparation, now you can request Amazon Product API!! The following code shows how to use SignedRequestsHelper. The key points of the examp...

Introduction In this post, I will explain how to switch view based on devices on the web application - like pc, smartphone, tablet. The basic strategy is checking user agenet which is sent from these devices, and building the specific html for the device. The key issue uis how to check user agent of all coming http requests efficiently and ellegantly. If you use some web application framework such as Java Spring Framework or PHP Symfony framework, the solution is using &qout;Filter&qout; which every http requests pass through. Let me show you the example implementation using Spring framework. Basic Strategy In filter check if "view_mode" in user cookies, set the view mode based on that cookie value if the cookie does not exist, set view mode based on user agent and publish corresponding view mode cookie set the selected view mode to HttpServletRequest attribute. In controller, set view template based on view mode in the HttpServletRequest attribute ...

In current software development world, serializing object to some human readable format such as "XML", "JSON" is quite common. In this post I will show you a small code snippet for encoding Java Object to XML and decoding XML to Java Object only using Java Standard API. I think this approach is suitable if you need to quick and simple solution to encoding and decoding java object to XML string. Actually code is very simple. Please see the code below :) If you need to input or output XML to file, you should pass FileOutputStream or FileInputStream instead of ByteArrayOutputStream or ByteArrayInputStream. package com.dukesoftware.utils.xml; import java.beans.XMLDecoder; import java.io.BufferedInputStream; import java.io.ByteArrayInputStream; import java.io.FileInputStream; import java.io.FileNotFoundException; public class XMLUtils { public final static Object decodeToXML(String file) throws FileNotFoundException{ try(XMLDecoder decoder =...

If you would like to join array with a separator, what will you do? e.g. Joining "a","b","c" with separator "," to "a,b,c". In PHP, you just use implode function. Actually PHP has a lot of functions like manipulating, converting string and array. In java, these kind of functions are not provided in official JDK. If you are familiar with Java Libraries, you just use them. - Joiner in google Guava, StringUtils in Apache Coommons. I will show you quick code snippet for jonining or imploding java string array with a separator. private static String join(String[] pieces, String separator) { if(pieces.length == 0) return ""; StringBuilder sb = new StringBuilder(pieces[0]); for(int i = 1; i < pieces.length; i++) { sb.append(separator).append(pieces[i]); } return sb.toString(); } If the number of array elements is zero, the method returns empty string, but if you would not prefer this...

Visual Studio 2013に乗り換えるためにVisual Studio 2010をアンインストールしようとしたところで問題発生。。。 先にVisual Studio 2010本体をアンインストールした後、Visual Studio 2010 SP1をアンインストールできなくて困っていました。 Google先生で調べたところ、以下の情報が見つかり、書かれていた通り実施したところ無事Visual Studio 2010 SP1のアンインストールに成功しました。 How to uninstall/remove Visual Studio SP1 アンインストール手順は以下の通りです。 http://www.microsoft.com/en-gb/download/details.aspx?id=23691 からMicrosoft Visual Studio 2010 Service Pack 1 (Installer)をダウンロード VS10sp1-KB983509.exeというファイル名でダウンロードされる(はず)。 コマンドプロンプトから以下のコマンドを実行 (以下の例は、c:\tempにVS10sp1-KB983509.exeがある場合) c:\temp\VS10sp1-KB983509.exe /uninstall /force ダイアログが立ち上がるので、アンインストールを選択して次へ進めばOK!

Have you ever treated XML only using library provided in Java SDK? I think many people use external library for treating XML in Java. To be honest, I would prefer to use JDom ;P But I think it's good thing to know how to treat XML only using Java SDK. I will show you small code snippet for treating XML especially from XPath usage. The classes you should remember for treating XML is: DocumentBuilderFactory DocumentBuilder Document XPathFactory XPath XPathExpression XPathConstants NodeList Node Oops, a bit quite too much. Anyway I will show you an example code for reading XML string and select nodes by XPath. package com.dulesoftware.xpath; import java.io.IOException; import java.io.StringReader; import javax.xml.parsers.DocumentBuilder; import javax.xml.parsers.DocumentBuilderFactory; import javax.xml.parsers.ParserConfigurationException; import javax.xml.xpath.XPath; import javax.xml.xpath.XPathConstants; import javax.xml.xpath.XPathExpression; import javax...