Introduction -History of IR- Components of IR – Issues –Open source Search engine Frameworks – The impact of the web on IR – The role of artificial intelligence (AI) in IR – IR Versus Web Search – Components of a Search engine- Characterizing the web.
UNIT II INFORMATION RETRIEVAL
Boolean and vector-space retrieval models- Term weighting – TF-IDF weighting- cosine similarity – Preprocessing – Inverted indices – efficient processing with sparse vectors – Language Model based IR – Probabilistic IR –Latent Semantic Indexing – Relevance feedback and query expansion.
UNIT III WEB SEARCH ENGINE – INTRODUCTION AND CRAWLING
Web search overview, web structure, the user, paid placement, search engine optimization/ spam. Web size measurement – search engine optimization/spam – Web Search Architectures – crawling – meta-crawlers- Focused Crawling – web indexes –- Near-duplicate detection – Index Compression – XML retrieval.
UNIT IV WEB SEARCH – LINK ANALYSIS AND SPECIALIZED SEARCH
Link Analysis –hubs and authorities – Page Rank and HITS algorithms -Searching and Ranking – Relevance Scoring and ranking for Web – Similarity – Hadoop & Map Reduce – Evaluation – Personalized search – Collaborative filtering and content-based recommendation of documents and products – handling “invisible” Web – Snippet generation, Summarization, Question Answering, Cross- Lingual Retrieval.
UNIT V DOCUMENT TEXT MINING
Information filtering; organization and relevance feedback – Text Mining -Text classification and clustering – Categorization algorithms: naive Bayes; decision trees; and nearest neighbor – Clustering algorithms: agglomerative clustering; k-means; expectation maximization (EM).