so, what do you think one needs to have inorder to be able have and support a personal search engine.
here's some of the things i think one needs - what additional stuff does one need - please comment
- Website/Domain - with all the SOPA, PIPA stuff and new laws coming
in wonder; if ".com" or ".org" or ".net" is good enough for a search
engine that's going to crawl websites and display some innovative
- HTML5 - for the frontend - or whatever - Please let everyone know if
you think of any other better alternative
- Python - for the backend - and probably other languages to support
server programming - and other stuff.
- Hardware - do you think its good enough to have the search-engine
hosted on some cloud hosting company - like Amazon - or - have one's
own cluster of machines and take care of the cluster yourself.
- NOTE: as data increases disk-space will increase exponentially.
- What's more easy to take care of - a Cloud or your own Network.
- Whats more cheap - Cloud or your own Network (cluster)
5. what else ???
- what kind of filesystem would you use to store such huge amounts of
data ? obviously has to be some kind of cluster filesystem - so that
you can support data availability
- what or which database would you use to store such huge data
- would you use open source software or buy software outright - what are the
implications of choosing one of the either ?
- How much money would be required to do all of the stuff mentioned
above ? - just some idea - so that everyone knows what to do for money
- how many developers would be required - whats the minimum number of
developers required to get a sample search engine ready for DEMO
- lets say you have 1 year to get the DEMO ready
- do you think 1 person can do it all
More questions will be added later
10 Apr '12, 16:11