Developing Framework of Web Scraper for Agriculture Data using Client Server Module
Prateeksha Nashipudi, Dr. S. B. Kulkarni, , ,
Affiliations 1:PG Scholar, SDM College of Engineering and Technology, Dharwad, Karnataka, India. 2:Associate Professor SDM College of Engineering and Technology, Dharwad, Karnataka, India.
Searching for relevant information becomes very difficult and sometimes we don’t find the exact information what we are actually seeking, so it results in time-consuming and repeating the same web page without knowingly. A system which will know our needs, requirements, preferences and patterns. This will retrieve the correct information and helps in fast processing. In this work, it is proposed that the personalized search engine for information retrieval system using the client-server module for user preferred information through intelligent search and storing the searched result in a database for further accessing of information is implemented. For information retrieval, a framework known as scrappy is used for retrieving all the user needed information by specifying the URL of that data. The fetched information is stored in the database. It helps in offline browsing, full-text search in the database and fast response and no repeating of web pages.
Prateeksha Nashipudi, Dr. S. B. Kulkarni."Developing Framework of Web Scraper for Agriculture Data using Client-Server Module". International Journal of Computer Engineering In Research Trends (IJCERT), ISSN:2349-7084, Vol.7, Issue 08, August - 2020,
Keywords : web scraping, crawling, user agent, scrapy framework and web spider.
 Alvarado, Antonio Cuahtlapantzi, Eduardo Vázquez Santacruz, and Mariano Gamboa Zúñiga. "Construction of a basic intelligent agent." Intelligent Systems Conference
(IntelliSys). IEEE, 2017.
 Pacheco-Reyes, Juan J., et al. "Multi-agent Architecture for User Adaptive Information Retrieval Systems." New Trends in Networking, Computing, Elearning,
Systems Sciences, and Engineering. Springer, Cham, 2015.
 Shankhdhar, Gaurav Kant, and Manuj Darbari. "Building custom, adaptive and heterogeneous multi-agent systems for semantic information retrieval using organizational-multi-agent systems engineering, O-MaSE." 2016 2nd International
Conference on Advances in Computing, Communication, & Automation (ICACCA).
 Singh, Aarti, and Anu Sharma. "A framework for semantics and agent based personalized information retrieval in agriculture." 2nd International Conference on Computing for Sustainable Global Development (INDIACom). IEEE, 2015.
 Nunes, Ingrid, et al. "Dynamically Adapting BDI Agent Architectures based on High level User Specifications." Computer Science Department, King’s College London(2010).  Luo, Junwei, and Xiao Xue. "Research on information retrieval system based on Semantic Web and multi-agent." International Conference on Intelligent Computing and Cognitive Informatics. IEEE, 2010.
 Czibula, Gabriela, et al. "IPA-An intelligent personal assistant agent for task performance support." 5th International Conference on Intelligent Computer Communication and Processing. IEEE, 2009.
 Liu, Lizhen, Shujing Wang, and Hantao Song. "Intelligent agents for cooperative designs in individual information retrieval." 8th International Conference on Computer Supported Cooperative Work in Design. Vol. 2. IEEE, 2004.
 Xu, Yuanzhong. "A model based on a three-layer agent of personalized information retrieval systems." International Conference on Image Analysis and Signal Processing. IEEE, 2011.
 THANGARAJ, MM. "Agent-Based Personalized Semantic Web Information Retrieval System." International Journal of Advanced Computer Science and Applications 5.8 (2014)
We have kept IJCERT is a free peer-reviewed scientific journal to endorse conservation. We have not put up a paywall to readers, and we do not charge for publishing. But running a monthly journal costs is a lot. While we do have some associates, we still need support to keep the journal flourishing. If our readers help fund it, our future will be more secure.