[TODO] 0. Find a template for the PhD proposal

That is straight forward. The task is just to look at other students PhD proposals also at some major conferences and see what kind of structure they use. A very common structure for papers is Jennifer Widom’s structure for writing a good research paper. This or a similar template will help to make the proposal readable in a good way. For this blog article I will follow Jennifer Widom more or less.

1. Write an Introduction

Here I will describe the use case(s) of a distributed graph data base. These could be

• indexing the web graph for a general purpose search engine like Google, Bing, Baidu, Yandex…
• running the backend of a social network like Facebook, Google+, Twitter, LinkedIn,…
• storing web log files and click streams of users
• doing information retrieval (recommender systems) in the above scenarios

There could also be very other use cases like graphs from

• biology
• finance
• regular graphs 
• geographic maps like road and traffic networks

2. Discuss all the related work

This is done to name all the existing approaches and challenges that come with a distributed graph data base. It is also important to set onself apart from existing frameworks like graph processing. Here I will name the at least the related work in the following fields:

• graph processing (Signal Collect, Pregel,…)
• graph theory (especially data structures and algorithms)
• (dynamic/adaptive) graph partitioning
• distributed computing / systems (MPI, Bulk Synchronous Parallel Programming, Map Reduce, P2P, distributed hash tables, distributed file systems…)
• redundancy vs fault tolerance
• network programming (protocols, latency vs bandwidth)
• data bases (ACID, multiple user access, …)
• graph data base query languages (SPARQL, Gremlin, Cypher,…)
• Social Network and graph analysis and modelling.

3. Formalize the problem of distributed graph data bases

After describing the related work and knowing the standard terminology it makes sense to really formalize the problem. Several steps have to be taken: There needs to be notation for distributed graph data bases fixed. This has to respect two things:
a) the real – so far unknown – problems that will be solved during PhD. In this way fixing the notation and formalizing the (unknown) problem will be kind of hard.
b) The use cases: For the web use case this will probably translate to scale free small world network graphs with a very small diameter. Probably in order to respect other use cases than the web it will make sense to cite different graph models e.g. mathematical models to generate graphs with certain properties from the related work.
The important step here is that fixing a use case will also fix a notation and help to formalize the problem. The crucial part is to choose the use case still so general that all special cases and boarder line cases are included. Especially the use case should be a real extension to graph processing which should of course be possible with a distributed graph data base. 
One very important part of the formalization will lead to a first research question:

4. Graph Query languages – Graph Algebra

I think graph data bases are not really general purpose data bases. They exist to solve a certain class of problems in a certain range. They seem to be especially useful where information of a local neighborhood of data points is frequently needed. They also often seem to be useful when schemaless data is processed. This leads to the question of a query language. Obviously (?) the more general the query language the harder to have a very efficient solution. The model of a relational algebra was a very successful concept in relational data bases. I guess a similar graph algebra is needed as a mathmatical concept for distributed graph data bases as a foundation of their query languages. 
Remark that this chapter has nothing much to do with distributed graph data bases but with graph data bases in general.
The graph algebra I have in mind so far is pretty similar to neo4j and consists of some atomic CRUD operations. Once the results are known (ether as an answer from the related work or by own research) I will be able to run my first experiments in a distributed environment. 

5. Analysis of Basic graph data structures vs distribution strategies vs Basic CRUD operations

As expected the graph algebra will consist of some atomic CRUD operations those operations have to be tested against all different data structures one can think of in the different known distributed environments over several different real world data sets. This task will be rather straight forward. It will be possible to know the theoretical results of most implementations. The reason for this experiment is to collect experimental experiences in a distributed setting and to understand what is really happening and where the difficulties in a distributed setting are. Already in the evaluation of graphity I realized that there is a huge gap between theoretical predictions and the real results. In this way I am convinced that this experiment is a good step forward and the deep understanding of actually implementing all this will hopefully lead to:

6. Development of hybrid data structures (creative input)

It would be the first time in my life where I am running such an experiment without any new ideas coming up to tweak and tune. So I am expecting to have learnt a lot from the first experiment in order to have some creative ideas how to combine several data structures and distribution techniques in order to make a better (especially bigger scaling) distributed graph data base technology.

7. Analysis of multiple user access and ACID

One important fact of a distributed graph data base that was not in the focus of my research so far is the part that actually makes it a data base and sets it apart from some graph processing frame work. Even after finding a good data structure and distributed model there are new limitations coming once multiple user access and ACID  are introduced. These topics are to some degree orthogonal to the CRUD operations examined in my first planned experiment. I am pretty sure that the experiments from above and more reading on ACID in distributed computing will lead to more reasearch questions and ideas how to test several standard ACID strategies for several data structures in several distributed environments. In this sense this chapter will be an extension to the 5. paragraph.

8. Again creative input for multiple user access and ACID

After heaving learnt what the best data structures for basic query operations in a distributed setting are and also what the best methods to achieve ACID are it is time for more creative input. This will have the goal to find a solution (data structure and distribution mechanism) that respects both the speed of basic query operations and the ease for ACID. Once this done everything is straight forward again.

9. Comprehensive benchmark of my solution with existing frameworks

My own solution has to be benchmarked against all the standard technologies for distributed graph data bases and graph processing frameworks.

10. Conclusion of my PhD proposal

So the goal of my PhD is to analyse different data structures and distribution techniques for a realization of distributed graph data base. This will be done with respect to a good runtime of some basic graph queries (CRUD) respecting a standardized graph query algebra as well as muli user access and the paradigms of ACID. 

11 Timetable and mile stones

This is a rough schedual fixing some of the major mile stones.

• 2012 / 04: hand in PhD proposal
• 2012 / 07: graph query algebra is fixed. Maybe a paper is submitted
• 2012 / 10: experiments of basic CRUD operations done
• 2013 / 02: paper with results from basic CRUD operations done
• 2013 / 07: preliminary results on ACID and multi user experiments are done and submitted to a conference
• 2013 /08: min 3 month research internship  in a company benchmarking my system on real data
• end of 2013: publishing the results
• 2014: 9 months of writing my dissertation

For anyone who has input, knows of papers or can point me to similar research I am more than happy if you could contact me or start the discussion!
Thank you very much for reading so far!

Filip: Even though I wasn’t agreeing with your Facebook skepticism in the beginning, I realized that I was overestimating Facebook in its promotion role. I’ve been creating extremely successful, targeted (extremely low budget though – just 5 or 10 euros at a time) campaigns for my band on Facebook.

Results?

Even though I managed to inflate the number of fans (with 12 fans/1 euro average), the interaction stayed the same. These campaigns aren’t entirely useless, though – having this number of fans or more looks nice in a smaller Croatian market and can help in booking bigger shows (thus getting to more fans)..but still dissappointing. I won’t campaign until we release a website/album/have something to sell.

Rene: I like your observation. The first was that gaining these paid likes does not really increase interactions and increase your reach. As I am saying to the In Legend guys all the time: “Money invested in facebook or even effort in facebook reach is not the best way to increase one’s reach” I am very glad that you came to the same conclusion and shared your insights!
Secondly I partially agree with the effect of large fan numbers while booking gigs. It certainly looks good to business partners like bookers, labels, distributers,… if your social media numbers burst. But again I would say the price is too high. With 1 Euro / 12 fans you would need to invest 1000 Euro for 12’000 fans an 12’000 isn’t even skyhigh (well I don’t know about croation standards). But as we know only a very small fraction of these 12’000 fans would actually become real fans and start interacting with you. All this for getting a gig! I guess this money could much better be invested in a high quality video which especially for a young band is a very good investment. With the video in combination with smart music downloads you will be able to increase your reach. Maybe not to 12’000 fans but still to a solid number of real fans that actually come to your concert because they really care! In this way your social media fancount (especially facebook) will also grow.

Filip: Question1 – sharing music!
About the thesis of providing music only on the band site – I think it’s hard for someone to become our fan if there is no music on Facebook. Choosing one song for preview and directing a fan to .com might work, but they are attracting entirely different audiences – think poppy indie rock vs. oriental, modern metal ballad. Is it okay to let this promo run free and spread like wildfire until the album release? Or should I provide 2/3 of songs for free, and ask for a mail adress for the 3rd one? This might be a good model.

Rene: I agree with what you say. At the time of writing the blog post you are referring too I wasn’t aware of the existing facebook music apps. The important thing is getting a sustainable contact to the person interested in your music. This is achieved ultimately by his email adress. But your question is very important. Of course you have to give people a bait. This could be

1. snippets
2. entire song(s) on streeming
3. a music video (In legend offers downloads under every music video)
4. a free download (without registration)

and I really don’t know where to set the boarder.
In the beginning times of In Legend we had 3 songs for streaming on myspace and 4 songs on the ep for download in exchange of an email adress. That turned out to be a good solution. People who liked the first songs where curious to download them together with one additional song. So I guess a 2/3 split would work as well. I will just warn you. Asking people for their mail address scares 4 of 5 people away. But hey at least you get the adresses of your fans that are really willing to give something for the music!
Now about the place where to make the connection. If you achive getting the fans mail adress via a smart Facebook music player or via download on your homepage I don’t care. Once people like your music (and chances are higher once you can talk to them frequently) they will also turn into facebook fans. So I recommend switching from rootmusic which you are using right now on your facebook profile to bandRX or Songpier since both services allow you to give access to your music in return of mail adresses. Songpier is a very new service but they also offer a cool mobile app (right now also without collecting mail adresses)
Here is a video about bandRX

Filip: Question2 – What do you think of Bandcamp?
I personally think it’s a great platform for selling music and there is an option of collecting mail adresses. The downside is that there is no valuable content that can be published, like blogs. It would be perfect if it was just a music-streaming, checkout widget on my site.

Rene: One of my Favourite (but retired) bands Jester’s Funeral have just published all their songs to bandcamp and have linked to their homepage and from the homepage to bandcamp. It is not quite the widged you are asking for but I guess this stays an option especially if you don’t bring the technical know how of programming a homepage that enables you to offer your music as a download in exchange of mail adresses.
There is only one thing that bothers me about bandcamp. They only let 200 fans per month download your music for free (email exchange) and offer a pay as much as you want option. From the money raised they keep 15% as a service charge. If you want more free downloads you have to buy them or have people pay for your music.
To some extend they offer a fair deal. It is a good service for a reasonable price. I as a programmer would just do it on my own have the full controll of my data and keep the 15% but I am pretty convinced that bandcamp should be a pretty good option for many musicians
I hope I could answer your questions to your satisfaction! Sorry that there isn’t always the clear black our white, right or wrong. Things are complex on the web but by reading your mail I am very convinced that you are asking the right questions which means that at least in online marketing you are far ahead of 95% of all musicians!

have a look at the app here:

Features and Problems

• works on facebook and on any other webpage
• enables setting more and more songs free for streaming while more people install the app (in order to spread the word)
• users need to connect (with facebook or via email adress) in order to listen
• some lightweight statistics
• encrypted flash player (not open source yet) that makes it hard to download the music (Though I myself have some moral problems with this kind of feature. But well it is how the industry works…)
• Slideshow of pictures to improve listening experience
• optimized usability for high conversion rates

The app runs on PHP, MySQL, JavaScript (MooTools) and you will need your own webspace in order to host it

A kind warning

The App was developed with a lot of time pressure and we had some nasty bugs that needed to be fixed. That is why the source code is messed up with some really fast and dirty quick fixes. Afterwards I never really had the time to clean up the source and make a good documentation. As my PhD progresses this situation will not change in the foreseeable future. Since my prediction says that Facebook will be overrun by Google+ within this year it is more than time to share the app!
The good part: most of the stuff can be reused once Google+ opens its API and the app can be transformed to a great social network.

Source code on google code

http://code.google.com/p/in-legend-facebook-music-streaming-app/source/checkout

Poster

Feel  free to download and look at my first poster with the Title:  a model for social news streams and time indices on graph data bases
You will probably not see so many things in it without the slides from my talk. So let me explain some things. I was looking into the data structures to model social news streams.  Basically there is the approach of normalized or denormalized relational data bases which I call the twitter approach for the reason that Twitter is doing something similar with FlockDB
I also looked into the case of saving the news stream as a flat file for every user in which the events from his friends are saved for every user. For some reason I thought I had picked up somewhere that facebook is running on such a system. But right now I can’t find the resource anymore. If you can, please tell me! Anyway while studying these different approaches I realized that the flat file approach even though it seems to be primitive makes perfect sense. It scales to infinity and is very fast for reading! Even though I can’t find the resource anymore I will still call this approach the Facebook approach.
I was now wondering how you would store a social news stream in a graph data base like neo4j in a way that you get some nice properties. More specifically I wanted to combine the advantages of both the facebook and the twitter approach and try to get rid of the downfalls. And guess what! To me this seems actually possible on graph data bases. The key Idea is to store the social network and content items created by the users not only in a star topology but also in a list topology ordered by time of occuring events. The crucial part is to maintain this topology which is actually possible in O(1) while Updates occure to the graph.

Talk

As mentioned together with this poster I was giving  a talk social news streams and time indices on social network graphs. Please feel free to download the slides. Unfortunally I improved the example while making the poster so that some pictures are not consistant with those from the poster! If I find the time I will not only update the slides but also give the talk as a video lecture on youtube! I think this will be helpful to spread the idea!

Future Work

1. I need to publish all these results in a good coference or journal
2. relevance filtering and recommendations which is the problem I am really interested in.
3. Implementing this stuff for my social network (see blog post)

Open Questions

1. Is it possible in neo4j to specify edgetypes (Relationship types) on runtime rather than compiletime.
2. If so: Is the time of accessing them O(1) with respect to the node degree?
3. If not: is there a graph data base that is capable of doing this?

Discussion

Anyway it is great to see how much more insight you get when thinking of a problem in a scientific way and not only implement it right away!

Answer all Questions in your Fan mail on a regularly basis (maybe weekly) on your website!

Answering fanmail one by one is really taking a lot of power. You could rather answer them publicly on your website. So if someone sends you an mail and puts some questions inside you will extract the question and post it in some section of your homepage. you order the questions and answers by date to show that you are actually answering questions regularly and thereby show that you are happy about more questions. you write something like

2011 August 30 – Bob asks: Hey band, I was always wondering could you do a show in ….
Musician from Band: Dear Bob thanks for your question. We would love to do a show in …

If there are more and more questions you can later on think about grouping them by topic and make it easier for your fans to navigate through them!
And by the way you can also extract questions from social media. Whenever you answer a question you should sent a quick reply to the person that asked the question notifying him or her that you answered it at the following link…

There are several reasons for this process:

1. For your fans it is really great and interesting content and they will visit your homepage more frequently. Especially because you also have several updates.
2. You don’t have to answer same questions several times. Which saves you time!
3. It shows that you actually answer to questions (especially if you post the date of the question!)
4. it is better than answering on facebook! because you controll the content and people are on your homepage (where one can also find your shop!)
5. it is content around your band name that will make your page also relevant for longtail queries for search queries. Which will in turn bring more natural traffic to your page.
6. Since you answer the question as a response it it still a pretty personal way to communicate.
7. you can later on refere to what you wrote!

I have seen an author of a book doing something similar 3 years ago and it worked perfect. She just answered 5 or 6 questions a month. I am excited to see what my bandmates from In Legend think about this idea!
By the way I am doing nothing else here in this blog. Earlier days if I had an idea I sent an mail to In Legend and later we discussed the idea and eather followed it or threw it away. That is time consuming. By blogging my ideas they are available for the future. If in Legend can’t follow the idea hopefully other people and bands find it useful! This is is a philosophy! The internet has shown in many great examples that open source and openess in general works amazingly fine!
The whole topic is by the way part of Customer relation management! I have never learned that in theory. So if you have some great resources on CRM (especially in music) I’ll be happy to get your reading recommendations.

• some differences between Facebook and Myspace.
• the filter functions of Facebook
• risks of using Facebook as the only marketing tool.
• how to handle and share your music online
• the importance of your homepage and newsletter
• and (of course) much more (-:

many Ideas have been already discussed in my blog. This article is just the first part of a series and there is more to come. Right now I am part of a project where I am collecting some interesting insights from youtube and more mature bands. So there will be more tips in my online music marketing section coming up soon!

Advertising on the rest of the web is still much more profitable than advertising on social networks.

If Facebook continues to attract advertisers, they could easily create an ad program that is similar to Google Adsense. Many Facebook-users provide the network with detailed information about their interests. Thus, Facebook will be able to deliver highly personalized advertising to any user on any website. They don’t even have to care about the websites content. Even if the user would not have a Facebook-account, Facebook could still be able to predict the topics of interest for users of a website and again deliver highly personalized ads that do not necessarily have to correspond to the content of the website. I strongly believe that this is the real thread coming from Facebook to Google and this is what Google is afraid of!
What is your oppinion? Why do you think Google had to enter the social networking market?

No Friendship Requests Necessary

There are no friendship requests in Google+. You just add people by email adress to circles. They will get a notification but if they don’t add you to one of their circles, they will not “follow” you. So the system is much like Twitter, where you decide who you are interested in and who not. This will probably prevent a lot of spam and take away all those nasty marketing opportunities to companies. I considered to be active on Google+ with my band but for what reason? Even if fans add a band to one of their circles it will probably not be the circle with their best friends that they will pay most attention to. But maybe we give it a try anyway!

No API Implemented Yet

What I am missing in Google+ right now is the opportunity to access this site via a strong API. I am sure that this will change once the site is successful.According to Rafe Needleman’s cnet Article the developer API will come once Google Plus is more mature.

The features and functions of Google+ will likely change substantially in short order. More functions will be definitely be added to the service, as well as increased integration with other Google apps. Giving developers access now might be premature, as some might built products that end up duplicating features that Google itself is just about to layer into the publicly available service.

Many Privacy Settings:

What other company but Google could know better about users’ privacy concerns. The network is centralized and not distributed. This of course is a huge privacy issue but one that most users don’t understand. For the average user who just wants to save his privacy in his social circle or extended social circle, Google+ is really great.

Great Usability Reminding of Apple:

The UI of Google Plus really is a strong thing. It just feels very good to use it. According to techcrunch especially the UIof circles goes back to no one less than Andy Hetzfeld one of the key designers of the first  Macintosh software.

Hetzfeld, who has been working at Google since 2005, is indeed the one we can thank for the better-looking interface on Google+, as he’s the design lead on the project. You’ll likely recognize his name from his time spent at Apple (1979 – 1984) where he was a key designer for the original Macintosh software team.

Response to Critique on other sites:

I read some blogs and articles saying that Google plus will not be able to compete with Facebook, that it is to late to enter the market and so on… Well, I would say that Google already has many advantages

• a huge user base
• fans
• a similar yet superior social networking product
• together with other google products they have a complete product
• android to push their social network to the mobile market
• a lot of experience on how to do things viral

I would also guess that the similarity to Facebook is intended. Great usability comes from consistency in design. It doesn’t take much time to learn from using Facebook to Google+. Some new things like circles have excellent usability and due to the similarity to the Facebook UI, the rest feels just easy to use. So this is not stealing or a lack of creativity. It is just the amount of perfection that you expect from a company like Google!
So join me and start inviting your friends to Google Plus. Here is how it works.