Comments on: Time lines and news streams: Neo4j is 377 times faster than MySQL

By: Mark Essel

Mark Essel — Fri, 19 Jul 2013 08:29:39 +0000

I’m still learning SQL but wouldn’t a single ordered query be faster?
SELECT ce.Text
FROM Entry e
JOIN Topic t ON t.ID = e.Topic_ID
JOIN UserBand ub ON ct.Band
ORDER by ub.User_ID
I could be totally off here, as it’s a little past 4am, and I don’t have the data or a shell to try this out

By: Mark Essel

Mark Essel — Fri, 19 Jul 2013 08:29:39 +0000

By: Junegunn Choi

Junegunn Choi — Tue, 19 Jul 2011 17:16:54 +0000

“How much time does it take to generate the stream for a single user?”
So it’s about latency of each request. It seems that the huge difference in the latency comes from the fact that Neo4j is embedded. (I’m just assuming that you’re using an embedded one, correct me if I’m wrong.) Cause it’s just much simpler to access the data in the same address space. However, scalability is an issue for embedded database. So I would love to see a follow-up experiment employing REST API of Neo4j server. I myself haven’t tried it yet.
Cheers.

By: Junegunn Choi

Junegunn Choi — Tue, 19 Jul 2011 17:16:54 +0000

By: Rene

Rene — Tue, 19 Jul 2011 16:11:46 +0000

as already said there is a lot of room to tweak the sql here. But I want to retrieve the stream for every user sperately. So as far as I understand your solution I think this won't be of any help. The question is: "How much time does it take to generate the stream for a single user?" Since this ist still pretty fast (especially with neo4j) I looped over all users so I could build an average and also have all different kind of user types included. If i would build a streaming system on SQL in a realworld I would just denormalize these tables and create one newsstream table that will just handle the updates and store date in a redundant way. the following two discussions on stack overview are helpfull. http://stackoverflow.com/questions/2835075/php-news-feed-database-design http://stackoverflow.com/questions/202198/whats-the-best-manner-of-implementing-a-social-activity-stream But I don't like a denormalized system. This is not flexible if the social graph of my network changes (e.g. edges are added or deleted) and it is also not very flexible if the functionalilty of the network grows.

By: Rene

Rene — Tue, 19 Jul 2011 16:11:46 +0000

By: Junegunn Choi

Junegunn Choi — Tue, 19 Jul 2011 15:19:20 +0000

Hi Rene,
I think your conclusion here is somewhat misleading.
I don’t think you should execute 854 SQLs (which is extremely inefficient), instead Users table should be included in the SQL above, so with one more join, you can traverse all the items with only one SQL. Considering the size of the data set, it shouldn’t take longer than a few seconds.

By: Junegunn Choi

Junegunn Choi — Tue, 19 Jul 2011 15:19:20 +0000

By: fengyan

fengyan — Tue, 19 Jul 2011 01:14:43 +0000

very nice and have a try
thanks

By: fengyan

fengyan — Tue, 19 Jul 2011 01:14:43 +0000

very nice and have a try
thanks