The goal of wikiminer is to create a specialized search engine for Wikipedia that is superior to a generic one such as Google's. Google's search engine cannot trust the data that it indexes while a specialized search engine for Wikipedia would be able to since Wikipedia is moderated. This implies that the specialized search engine could be simpler and have superior performance.
There are many possible approaches to data mining Wikipedia. The approach that I will focus on is analyzing the link structure instead of analyzing the text itself.
Additionally, instead of creating a traditional search engine where a user queries for a topic and the search engine returns articles with information on that topic, the search engine that I am attempting to create will ask the user for two different topics and return articles that includes information on the relationship between the two topics.
The project will consist of two parts:
1. Parsing the Wikipedia dumps into a graph so that it is more suitable for analysis
2. Designing and implementing an algorithm that will run with the above created graph as the input.
2 comments:
look for literature-related-discovery or for literature-based-discovery. they implemented similar ideas to your proposed search engine , and this technique seems to be a way to lead people to new innovations.
Hi,
Great effort.
Have you seen Powerset?
http://www.powerset.com/
We can carry on the conversation through email at kq@kquee.com :)
Post a Comment