Comments on: Metalcon finally gets a redesign – Thinking about high scalability https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/ Extract knowledge from your data and be ahead of your competition Tue, 17 Jul 2018 11:07:57 +0000 hourly 1 https://wordpress.org/?v=4.9.6 By: Aurelius Titan graph enables realtime querying with 2400 concurrent users on a distributed graph database! https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33494 Wed, 21 Aug 2013 13:08:56 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33494 […] the redesign phase of metalcon we started playing around with HBase to support the architecture of our like button and especially […]

]]>
By: Aurelius Titan graph enables realtime querying with 2400 concurrent users on a distributed graph database! https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33496 Wed, 21 Aug 2013 13:08:56 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33496 […] the redesign phase of metalcon we started playing around with HBase to support the architecture of our like button and especially […]

]]>
By: My ranked list of priorities for Backend Web Programming: Scalability > Maintainable code > performance https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33492 Sat, 27 Jul 2013 09:22:06 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33492 […] I had in mind that performace is the same as scaling (which is a wrong assumption) and asked about Ruby on rails vs GWT. I am totally convinced that GWT is much more performant than ruby. But I have seen GWT code and it […]

]]>
By: My ranked list of priorities for Backend Web Programming: Scalability > Maintainable code > performance https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33493 Sat, 27 Jul 2013 09:22:06 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33493 […] I had in mind that performace is the same as scaling (which is a wrong assumption) and asked about Ruby on rails vs GWT. I am totally convinced that GWT is much more performant than ruby. But I have seen GWT code and it […]

]]>
By: René Pickhardt https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33489 Wed, 17 Jul 2013 16:44:02 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33489 Hey Ruben,
thanks again for our phone conversation and now for your amazing summary! I was already more and more convinced to use Ruby on Rails for the Framework since I really see some advantages and the issues that will prevent me from scaling will be implemented as stand alone services and integrated later on.
The Ruby discussion is not quite through yet but still your comment pushed it in a certain direction because it really shows how the community works!
best Rene

]]>
By: René Pickhardt https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33490 Wed, 17 Jul 2013 16:44:02 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33490 Hey Ruben,
thanks again for our phone conversation and now for your amazing summary! I was already more and more convinced to use Ruby on Rails for the Framework since I really see some advantages and the issues that will prevent me from scaling will be implemented as stand alone services and integrated later on.
The Ruby discussion is not quite through yet but still your comment pushed it in a certain direction because it really shows how the community works!
best Rene

]]>
By: Why would musicians use online social networking sites? https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33486 Wed, 17 Jul 2013 09:56:49 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33486 […] been running metalcon an online social network for metal fans and metal bands. As written recently I have the the chance to rewrite the entire platform with a team of 6 programmers. This time we want to do it the correct why. Instead of Thinking of […]

]]>
By: Why would musicians use online social networking sites? https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33487 Wed, 17 Jul 2013 09:56:49 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33487 […] been running metalcon an online social network for metal fans and metal bands. As written recently I have the the chance to rewrite the entire platform with a team of 6 programmers. This time we want to do it the correct why. Instead of Thinking of […]

]]>
By: Ruben https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33483 Thu, 27 Jun 2013 02:51:46 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33483 Rene,
Thanks for all of your assistance! Let me recap and add to our recent conversation, and make a case for Ruby on Rails (and / or other RAD frameworks – I don’t want ignite a Rails vs Django vs Cake debate – any of these frameworks share some fo the same pros and cons).
I do not claim to be an expert, and welcome any corrections . I wrote this on a plane with no internet access so, keep that in mind.
Pros
1. Rails Scalability: Ruby on Rails and other frameworks have come a long ways in terms of performance and scaling.. It’s true that older versions of Rails had issues, which have since been improved or resolved. As far as blocking is concerned, app servers like Passenger Phusion spread out processes over cores to resolve this issue. I list this as a pro, though it could be a disadvantage as well, to point out things have improved for Rails in this area.
2. Plugins, Gems, APIs etc: Frameworks offer features such as libraries, plug ins, GEMS (Ruby on Rails) that can help cut development time down. Autentication, administration, security, and database GEMS are available in Ruby on Rails so you can focus on what is different about your application. (For example, if you want to autenticate with Facebook, you could write your own code, or you could drop in a GEM that has the code to do it. That lets you focus on the rest of your app for which there are no plugs-ins.)
3. RailsCasts – 300 episodes that really help you to learn Ruby on Rails. One of the best resources for learning how to do things with ROR.
4. Proven: Twitter, Hulu, Linked In are examples of sites that started with Rails. True some of them have removed rails from the back end, but ROR gave them a competitive advantage and allowed them to grow in the first place. (Also, the versions of Rails they used had more scalablity issues than the current version does).. Unless your site is going to have tens or hundreds of millions of users, the odds that you will not need to remove rails from the back end. So in a nutshell, ROR is a proven framework for quickly creating top tier apps.
5. Convention not configuration: Instead of worring so much about configuration, simply write code to get the job done.
6. Ease of development: Once you understand ROR, it’s very quick to develop functionaly. For example, I’ve created social network follow functionality in Neo4j & ROR in a little under 10 minutes.
7. Lightweight frameworks: Don’t need the heavy functionality of RAILS? Need a lighter library to host web services? Try Sinatra instead of the full Rails stack.. Create sockets, live chatting, etc this way..
8. Proven in hybrid systems: (Both an advantage and disadvantage.) To scale out, no matter what you write your system in, the odds are you will be using a hybrid design. Django, Ruby on Rails, PHP with Cake, Java etc., node.js, etc you’ll probly have a hybrid system with in memory database servers, Node.js, message queuing systems, app servers, data sharding, data replication, HTTP caching, load balancers etc. The advantage rails offers is the experience others have with it – they have shared it and provided a road map so you don’t have to guess at how to design a ROR hybrid.. Disadvantage is that the architecutre of a hybrid sytem is much more complex..
9. Framework: There are common sets of functions to use, a common architecture, etc.. Compare this to Node.JS.. Node.js is not an app framework – you don’t have a robust set of standard libraries to call on for common application functionality. Node.js doesn’t help you develop applications faster, though it will help you develop faster applications and to split out web services with greate ease.
10. JavaScript and Jquery: Rails now uses Jquery and Javascript as standards. If you want to develop even faster, Coffescript can be used to develop your javascript routines. Take advantage of the many query routines already written (like endless pages / lazy loading of pages) to help speed up your development. Integrate Twitter Bootstrap with ROR.. Even use GWT with ROR as well … The cons of this is you have to learn how to integrate these with Rails, but there are GEMS to handle that so you don’t have to! Once you get past the intial learning curve of using these with Rails, it’s pretty easy..
11. Learning curve: ROR in many ways is easy to learn. Scaffolding is a great way to start creating an app.. I still use it even though I don’t need to use it anymore to develop rails apps. Describing a table with minimal typing (you don’t specify field lengths etc – but you can if you want to) via scaffolding also creates the model and views for you to help get your app up and running very quickly.. Very quick to develop this way, and easy to learn.
12. Database migrations: Add or remove fields to your tables with migrations. Very useful in synchronizeing test and production database schemas – ROR will make the necessary changes to production tables without the assistance of a DBA.
13. DBA is not Required: A database administrator (DBA) is not necessary to develop ROR apps. Well, a DBA could be helpful, but it’s not necessary to have a DBA create tables etc.. Rails will do this for the developer, resulting in quicker development time.
14. Streaming: Stream data to create live chatting, or send data to mobile devices with long request lifetimes.. While it’s not as robost as doing so with Node.js, nevertheless it offers node.js like functionality and performance. (Available with ROR 4 and with 3.2 with some tweaks.)
15. JRuby.. Uses a Java VM and can access Java APIs.
16. Community: The Rails community offers a tremendous ammount of support not often found with other frameworks. Have a problem? It’s easy to get help from the community. Like this response about scalability – the community is very helpful.
Disadvantages.
1. ORM: Well, Rails uses Active Record, which has some significant overhead. On the other hand it’s possible to use sql statements and procs so that performance can be optimized. With my current Neo4j project, I’m not using ActiveRecord – only Activemodel, and there are other alternatives to Activerecord. While some say never use ORM due to scalability, I think ORM’s proper place is to quickly develop an app – If a project is successful, there will be time and money to move to something else.. If a project is only moderately succesful, you may find that an ORM is just fine.
2. Threading / Thread blocking: It’s possible to get around this, and Rails 4 offers tread safe functionality, but still, it’s something that has to be considered in terms of the server architecture and app enviornment (Passenger Phusion etc)
The typical way to get around this typically involves data sharing, data replication, load balancing, caching, in memory database servers (Redis, Orient, MongoDB etc). Of course, you do want to make sure your code and database design take scalability into account – so use best practices when designing and coding.
4. Code Samples: Not so much a ROR issue but more of a community issue, some examples of code don’t work in the version of rails you may use. Or the examples are outdated… Some GEMS won’t work with new verions of rails, or are minimally documented. Sometimes you might find that you could end up spending more time trying to get the GEM to work than simply writing the equivalent code yourself. (If I am having problems witha a new GEM, and can’t get it working in 3 hours or less, I just remove the GEM and start to code the functionality myself.)
5. Performance issues are resolved addressed by adding more servers. The way to scale out sites based on frameworks is to use multiple web servers to handle high loads. Node.js does not require this techniqe in terms of web server, at least not to the same degre, but node.js doesn’t provide the framework for apps and websites either.
6. Plugins, Gems, APIs etc: Frameworks offer features such as libraries, plug ins, GEMS (Ruby on Rails) that can help cut development time down. Autentication, administration, security, and database GEMS are available in Ruby on Rails so you can focus on what is different about your application.
7. Lack of a standard IDE.. While you can use Eclipse or Netbeans (or some others), there isn’t a default editor or IDE for ROR. I edit with Textmate.. I could use Eclipse, which I love, but for now I’m happy with using a text editor. But not everyone likes to code this way..
8. Limited Windows support: I started programming in ROR on Windows, but occassionally I would come across something that had not yet been compiled to Windows.. I’m not interested in wasting time trying to complile something or searching for a workaround for Windows – I want to get my project running and developed quickly.. Things may have improved, but I ended up switching to OSX where I’ve never had this issue.
9. Framework Scalability: If you expect Rails or just about any other framework to scale right out of the box, it won’t. Not to the levels you will need to have a twitter or facebook. You’ll have to create a hybrid architecture to do this.
10. Heavy Metal: As with most framework based apps, you add metal (servers) to help scale. Node.js is very fast, and you don’t have to throw so much metal at an app. The tradeoff with Node.js is that it will take longer to develop your apps.
11. MVC. If one does not like like the MVC pattern, ROR might not be a good choice. I believe it’s possible to use other patterns, but out of the box it’s MVC and the examples and help provided online will primarily consist of MVC…
12. REST & routing. If you don’t like REST, and don’t want to be confined to it, rails might not be a good choice. The routing is very flexible, and REST can have additional methods besides edit/delete, etc, but out of the box this is how it is and if you come from an environment that doesn’t use REST, you might not like it so much.
There are many other pros and cons.. I’m partial to ROR.. I’ve done JSP, ASP, ASP.NET, Websphere, and I just find ROR faster to develop with. I’ll pick up Djanogo soon – I’m an equal opportunity coder when it comes to earning money so I *will* develop with any system/framework/etc.
Finally, there are some things that irritate me, and some things I love.. But all in all I highly recommend it.

]]>
By: Ruben https://www.rene-pickhardt.de/metalcon-finally-becomes-a-redesign-thinking-about-high-scalability/#comment-33484 Thu, 27 Jun 2013 02:51:46 +0000 http://www.rene-pickhardt.de/?p=1631#comment-33484 Rene,
Thanks for all of your assistance! Let me recap and add to our recent conversation, and make a case for Ruby on Rails (and / or other RAD frameworks – I don’t want ignite a Rails vs Django vs Cake debate – any of these frameworks share some fo the same pros and cons).
I do not claim to be an expert, and welcome any corrections . I wrote this on a plane with no internet access so, keep that in mind.
Pros
1. Rails Scalability: Ruby on Rails and other frameworks have come a long ways in terms of performance and scaling.. It’s true that older versions of Rails had issues, which have since been improved or resolved. As far as blocking is concerned, app servers like Passenger Phusion spread out processes over cores to resolve this issue. I list this as a pro, though it could be a disadvantage as well, to point out things have improved for Rails in this area.
2. Plugins, Gems, APIs etc: Frameworks offer features such as libraries, plug ins, GEMS (Ruby on Rails) that can help cut development time down. Autentication, administration, security, and database GEMS are available in Ruby on Rails so you can focus on what is different about your application. (For example, if you want to autenticate with Facebook, you could write your own code, or you could drop in a GEM that has the code to do it. That lets you focus on the rest of your app for which there are no plugs-ins.)
3. RailsCasts – 300 episodes that really help you to learn Ruby on Rails. One of the best resources for learning how to do things with ROR.
4. Proven: Twitter, Hulu, Linked In are examples of sites that started with Rails. True some of them have removed rails from the back end, but ROR gave them a competitive advantage and allowed them to grow in the first place. (Also, the versions of Rails they used had more scalablity issues than the current version does).. Unless your site is going to have tens or hundreds of millions of users, the odds that you will not need to remove rails from the back end. So in a nutshell, ROR is a proven framework for quickly creating top tier apps.
5. Convention not configuration: Instead of worring so much about configuration, simply write code to get the job done.
6. Ease of development: Once you understand ROR, it’s very quick to develop functionaly. For example, I’ve created social network follow functionality in Neo4j & ROR in a little under 10 minutes.
7. Lightweight frameworks: Don’t need the heavy functionality of RAILS? Need a lighter library to host web services? Try Sinatra instead of the full Rails stack.. Create sockets, live chatting, etc this way..
8. Proven in hybrid systems: (Both an advantage and disadvantage.) To scale out, no matter what you write your system in, the odds are you will be using a hybrid design. Django, Ruby on Rails, PHP with Cake, Java etc., node.js, etc you’ll probly have a hybrid system with in memory database servers, Node.js, message queuing systems, app servers, data sharding, data replication, HTTP caching, load balancers etc. The advantage rails offers is the experience others have with it – they have shared it and provided a road map so you don’t have to guess at how to design a ROR hybrid.. Disadvantage is that the architecutre of a hybrid sytem is much more complex..
9. Framework: There are common sets of functions to use, a common architecture, etc.. Compare this to Node.JS.. Node.js is not an app framework – you don’t have a robust set of standard libraries to call on for common application functionality. Node.js doesn’t help you develop applications faster, though it will help you develop faster applications and to split out web services with greate ease.
10. JavaScript and Jquery: Rails now uses Jquery and Javascript as standards. If you want to develop even faster, Coffescript can be used to develop your javascript routines. Take advantage of the many query routines already written (like endless pages / lazy loading of pages) to help speed up your development. Integrate Twitter Bootstrap with ROR.. Even use GWT with ROR as well … The cons of this is you have to learn how to integrate these with Rails, but there are GEMS to handle that so you don’t have to! Once you get past the intial learning curve of using these with Rails, it’s pretty easy..
11. Learning curve: ROR in many ways is easy to learn. Scaffolding is a great way to start creating an app.. I still use it even though I don’t need to use it anymore to develop rails apps. Describing a table with minimal typing (you don’t specify field lengths etc – but you can if you want to) via scaffolding also creates the model and views for you to help get your app up and running very quickly.. Very quick to develop this way, and easy to learn.
12. Database migrations: Add or remove fields to your tables with migrations. Very useful in synchronizeing test and production database schemas – ROR will make the necessary changes to production tables without the assistance of a DBA.
13. DBA is not Required: A database administrator (DBA) is not necessary to develop ROR apps. Well, a DBA could be helpful, but it’s not necessary to have a DBA create tables etc.. Rails will do this for the developer, resulting in quicker development time.
14. Streaming: Stream data to create live chatting, or send data to mobile devices with long request lifetimes.. While it’s not as robost as doing so with Node.js, nevertheless it offers node.js like functionality and performance. (Available with ROR 4 and with 3.2 with some tweaks.)
15. JRuby.. Uses a Java VM and can access Java APIs.
16. Community: The Rails community offers a tremendous ammount of support not often found with other frameworks. Have a problem? It’s easy to get help from the community. Like this response about scalability – the community is very helpful.
Disadvantages.
1. ORM: Well, Rails uses Active Record, which has some significant overhead. On the other hand it’s possible to use sql statements and procs so that performance can be optimized. With my current Neo4j project, I’m not using ActiveRecord – only Activemodel, and there are other alternatives to Activerecord. While some say never use ORM due to scalability, I think ORM’s proper place is to quickly develop an app – If a project is successful, there will be time and money to move to something else.. If a project is only moderately succesful, you may find that an ORM is just fine.
2. Threading / Thread blocking: It’s possible to get around this, and Rails 4 offers tread safe functionality, but still, it’s something that has to be considered in terms of the server architecture and app enviornment (Passenger Phusion etc)
The typical way to get around this typically involves data sharing, data replication, load balancing, caching, in memory database servers (Redis, Orient, MongoDB etc). Of course, you do want to make sure your code and database design take scalability into account – so use best practices when designing and coding.
4. Code Samples: Not so much a ROR issue but more of a community issue, some examples of code don’t work in the version of rails you may use. Or the examples are outdated… Some GEMS won’t work with new verions of rails, or are minimally documented. Sometimes you might find that you could end up spending more time trying to get the GEM to work than simply writing the equivalent code yourself. (If I am having problems witha a new GEM, and can’t get it working in 3 hours or less, I just remove the GEM and start to code the functionality myself.)
5. Performance issues are resolved addressed by adding more servers. The way to scale out sites based on frameworks is to use multiple web servers to handle high loads. Node.js does not require this techniqe in terms of web server, at least not to the same degre, but node.js doesn’t provide the framework for apps and websites either.
6. Plugins, Gems, APIs etc: Frameworks offer features such as libraries, plug ins, GEMS (Ruby on Rails) that can help cut development time down. Autentication, administration, security, and database GEMS are available in Ruby on Rails so you can focus on what is different about your application.
7. Lack of a standard IDE.. While you can use Eclipse or Netbeans (or some others), there isn’t a default editor or IDE for ROR. I edit with Textmate.. I could use Eclipse, which I love, but for now I’m happy with using a text editor. But not everyone likes to code this way..
8. Limited Windows support: I started programming in ROR on Windows, but occassionally I would come across something that had not yet been compiled to Windows.. I’m not interested in wasting time trying to complile something or searching for a workaround for Windows – I want to get my project running and developed quickly.. Things may have improved, but I ended up switching to OSX where I’ve never had this issue.
9. Framework Scalability: If you expect Rails or just about any other framework to scale right out of the box, it won’t. Not to the levels you will need to have a twitter or facebook. You’ll have to create a hybrid architecture to do this.
10. Heavy Metal: As with most framework based apps, you add metal (servers) to help scale. Node.js is very fast, and you don’t have to throw so much metal at an app. The tradeoff with Node.js is that it will take longer to develop your apps.
11. MVC. If one does not like like the MVC pattern, ROR might not be a good choice. I believe it’s possible to use other patterns, but out of the box it’s MVC and the examples and help provided online will primarily consist of MVC…
12. REST & routing. If you don’t like REST, and don’t want to be confined to it, rails might not be a good choice. The routing is very flexible, and REST can have additional methods besides edit/delete, etc, but out of the box this is how it is and if you come from an environment that doesn’t use REST, you might not like it so much.
There are many other pros and cons.. I’m partial to ROR.. I’ve done JSP, ASP, ASP.NET, Websphere, and I just find ROR faster to develop with. I’ll pick up Djanogo soon – I’m an equal opportunity coder when it comes to earning money so I *will* develop with any system/framework/etc.
Finally, there are some things that irritate me, and some things I love.. But all in all I highly recommend it.

]]>