THE PRICE OF RELIABILITY IS THE PURSUIT OF THE UTMOST SIMPLICITY.
On how to save time by building a Proof of concept.
Magento is huge in size (500 Mb!) and can be very slow.
This particular Magento webshop contained only about 3000 product variations. On a good day, it took at least 5 seconds for the search results to appear, even with one visitor on the site. This was acknowledged as a serious problem, since slow page speeds can easily cause customers to leave the webshop out of frustration.
Out of curiosity, I just wanted to know how fast this search process theoretically could be, once you simply aggregate the product data. So I build a Proof of Concept, during which I noticed I was overcomplicating.
A Proof of Concept generally needs no real data, no database, no fancy user-interface and no internet connection.
On the basis of the product array, I wrote two programs:
1) A small program to generate 22000 random product variations. Each product variation had ten semi-random characteristics like brand, price, size, category and weight. These random products were stored in a simple CSV file.
2) Then I wrote a program to create a user interface, scan the CSV file and select and show the qualifying product variations.
However, this PoC was never intended to be integrated into Magento. The company had to rent many web servers (application/DB) at AWS to speed up the site and to accommodate for traffic peaks during special sales and right after sending newsletters. Total hosting costs were probably in the range of $5000 to $10000 per month! However, with proper software, even a shared(!) hosting account (at $10 a month), could have done the job much quicker!