RPC – Data Science, Data Analytics and Machine Learning Consulting in Koblenz Germany

Building an Autocomplete Service in GWT screencast Part 4: Integrating the neo4j Data base

Rene — Thu, 20 Jun 2013 12:38:46 +0000

In this screencast of my series I explain at a very basic level how to integrate a data base to pull data for autocomplete queries. Since we have been working with neo4j at this time I used a neo4j data base. It will be only in the next two parts of this series where I introduce an efficient way of handling the data base (using the context listener of the web server) and building fast indices. So in this lesson the resulting auto complete service will be really slow and impractical to use but I am sure for didactic reasons it is ok to invest 7 minutes for a rather bad design.
Anyway if you want to use the same data set as I used in this screencast you can go to http://data.related-work.net and find the data set as well as a description of the data base schema:

Building an Autocomplete Service in GWT screencast Part 3: Getting the Server code to send a basic response

Rene — Mon, 17 Jun 2013 12:20:11 +0000

In this screencast of my series on building an autocomplete service you will learn how to implement a Server servlet in GWT such that autocomplete queries receive a response. In this video the response will always be static and very naive. It will be up to the fourth part of this series which will follow already this week to make the server to something meaningful with the query. This part is rather created to see how the server is supposed to be invoked and what kind of tools and classes are needed. So see this as a preparation for the really interesting stuff.

If you have any questions, suggestions and comments feel free to discuss them.

Building an Autocompletion on GWT screencast Part 2: Invoking The Remote Procedure Call

Rene — Tue, 12 Mar 2013 07:25:00 +0000

Hey everyone after posting my first screencast in this series reviewing the basic process for creating remote procedure calls in GWT we are now finally starting with the real tutorial for building an autocomplete service.
This tutorial (again hosted on wikipedia) covers the basic user interface meaning

how to integreate a SuggestBox instead of a textfield into the GWT Starter project
how to set up the neccessary stuff (extending a SuggestOracle) to fire a remote procedure call that requests suggestions if the user has typed something.
how to override the necessary methods from the SuggestOracle Interface

So here we go with the second part of the screencast which you can of course directly download from wikipedia:

Feel free to ask questions, give comments and improve the screencast!

Building an Autocompletion on GWT screencast Part 1: Getting Warm – Reviewing remote procedure calls

Rene — Tue, 19 Feb 2013 09:11:29 +0000

Quite a while ago I promised to create some screencasts on how to build a (personalized) Autocompletion in GWT. Even though the screencasts have been created for quite some time now I had to wait publishing them for various reasons.
Finally it is now the time to go public with the first video. I do really start from scratch. So the first video might be a little bit boaring since I am only reviewing the Remote Procedure calls of GWT.
A litte Note: The video is hosted on Wikipedia! I think it is important to spread knowledge under a creative commons licence and the youtubes, vimeos,… of this world are rather trying to do a vendor lock in. So If the embedded player is not so well you can go directly to wikipedia for a fullscreen version or direct download of the video.

Another note: I did not publish the source code! This has a pretty simple reason (and yes you can call me crazy): If you really want to learn something, copying and pasting code doesn’t help you to get the full understanding. Doing it step by step e.g. watching the screencasts and reproducing the steps is the way to go.
As always I am open to suggestions and feedback but please have in mind that the entire course of videos is already recorded.

Building an Autocompletion on GWT with RPC, ContextListener and a Suggest Tree: Part 0

Rene — Wed, 13 Jun 2012 13:15:29 +0000

Over the last weeks there was quite some quality programming time for me. First of all I built some indices on the typology data base in which way I was able to increase the retrieval speed of typology by a factor of over 1000 which is something that rarely happens in computer science. I will blog about this soon. But heaving those techniques at hand I also used them to built a better auto completion for the search function of my online social network metalcon.de.
The search functionality is not deployed to the real site yet. But on the demo page you can find a demo showing how the completion is helping you typing. Right now the network requests are faster than google search (which I admit it is quite easy if you only have to handle a request a second and also have a much smaller concept space). Still I was amazed by the ease and beauty of the program and the fact that the suggestions for autocompletion are actually more accurate than our current data base search. So feel free to have a look at the demo:
http://134.93.129.135:8080/wiki.html
Right now it consists of about 150 thousand concepts which come from 4 different data sources (Metal Bands, Metal records, Tracks and Germen venues for Heavy metal) I am pretty sure that increasing the size of the concept space by 2 orders of magnitude should not be a problem. And if everything works out fine I will be able to test this hypothesis on my joint project related work which will have a data base with at least 1 mio. concepts that need to be autocompleted.
Even though everyting I used but the ContextListener and my small but effective caching strategy can be found at http://developer-resource.blogspot.de/2008/07/google-web-toolkit-suggest-box-rpc.html and the data structure (suggest tree) is open source and can be found at http://sourceforge.net/projects/suggesttree/ I am planning to produce a series of screencasts and release the source code of my implementation together with some test data over the next weeks in order to spread the knowledge of how to built strong auto completion engines. The planned structure of these articles will be:

part 1: introduction of which parts exist and where to find them

Set up a gwt project
Erease all files that are not required
Create a basic Design

part 2: AutoComplete via RPC

Neccesary client side Stuff
Integration of SuggestBox and Suggest Oracle
Setting up the Remote procedure call

part 3: A basic AutoComplete Server

show how to fill data with it and where to include it in the autocomplete
disclaimer! of not a good solution yet
Always the same suggestions

part 4: AutoComplete Pulling suggestions from a data base

inlcuding a data base
locking the data base for every auto complete http request
show how this is a poor design
demonstrate low response times speed

part 5: Introducing the context Listener

introducing a context listener.
demonstrate lacks in speed with every network request

part 6: Introducing a fast Index (Suggest Tree)

inlcude the suggest tree
demonstrate increased speed

part 7: Introducing client side caching and formatting

introducing caching
demonstrate no network traffic for cached completions

not covered topics: (but for some points happy for hints)

on user login: create personalized suggest tree save in some context data structure
merging from personalized AND gobal index (google will only display 2 or 3 personalized results)
index compression
schedualing / caching / precalculation of index
not prefix retrieval (merging?)
css of retrieval box
parallel architectures for searching