Data Extraction Based on Web Scrapy

In recent years, with the growth of big data the information on the Internet is increasing rapidly, it has become important to have technologies that help the user to obtain the information efficiently and easily. Many undergraduate students buy books and school supplies through online stores such as Amazon and other websites. How to effectively obtain information from the websites. This paper implements a Python web scraper to extract the data from the target websites and store the collected data on the comma separated values file. This system aims to collect information about the products that undergraduate students need from the target websites and return them to the users with a simple page. Users can search for product that they are want to obtain information about it and the crawler crawled the following information (product name, product price, product URL) from the following online stores: Amazon, eBay, Jarir, and Extra then stored it in the comma-separated values file for information analysis. The data scraped by the crawler and saved on the comma-separated values file is 9083 records. The importance of the system lies in its effective ability to reach products fast and high efficiency and enable price comparison of similar products on different online shopping stores therefore saving the searching time.