Abstract:
The project’s main objective was to design and implement a system for gathering data from
the Polish real estate market that crawls websites every few minutes and downloads tens of
thousands of pages every month so that live listings can be displayed in a user-friendly way.
It was integrated with a web interface for data representation and user interaction.
Moreover, to achieve reliability, agility and scalability, we made our project cloud-native by
design.
The paper contains a description of the offers’ aggregator - we will discuss the mechanism
behind it, the other tools, and all of the different approaches to gathering data from the
internet.
Next, we will move to the web interface for data representation and User Interface design.
Last but not least, we will discuss deployment in a cloud environment - how it works, what
approach we have chosen, how DevOps culture influences the project, and finally, why
Cloud solutions helped us to be one step ahead of the competitors.
