Search engine question

mac_mogul · 01-20-2004, 08:07 PM

Something I've always wondered about, How exactly does a search engine work?

Larry · 01-20-2004, 10:11 PM

Lets see if I can sum up the key areas of a search engine:

1) A user enters a key word or key phrase
2) The "search engine" then "returns a result set" of "links" that the search engine "thinks" is "most relevant" to the key word or phrase

Now, that was easy, I will now try to explain the meanings of the words in quotes

"search engine"- obviously there are the main 3 (google, yahoo, msn), but a search engine is an application that has a simple front-end interface and a HUGE database of words and websites.

"returns a result set"- is a list of websites ranked in order based upon the search engine's "algorithm"

"algorithm"- each search engine has its own special code or algorithm that sorts or ranks the database full of websites. Needless to say, the algorithm is VERY, VERY complex. Its made of thousands and thousands of variables that determine the sort order for key words entered into the search engine.

"links"- basically each link is a website that has been indexed or spidered or inserted into the search engine's database

"thinks"- the search engine's brains, if you will, is the algorithm. It has rules and values that help determine the sort order of listing a result set.

"most relevant"- again, determined by the algorithm, the search engine's job is to give you a list of websites IT THINKS is what you want. Sometimes it works. Sometimes the search engine fails to return to the user, what the user is looking for.

hope that helps!

T1junkie · 01-21-2004, 08:49 AM

search engines have special programs called spiders that are constantly searching around the web for new pages and logging the addresses

thats what the meta tags in the HTML soiurce code are for (if you have ever looked)

mac_mogul · 01-22-2004, 12:57 PM

Hmm, I thought it seemed too easy...

And wow larry, that's knowledge at it's best right there! Incredible that you know this kinda stuff! And thanks for the answer, it clears things up alot! For the longest time I thought it was just like some sort of magic-technology thing...w

w...

Vicious · 01-25-2004, 06:52 PM

How Internet Search Engines Work
http://computer.howstuffworks.com/search-engine.htm

Larry · 01-26-2004, 02:34 AM

Most major search engines do not really use meta-tags, at least in the way you mean.

Even if a search engine does use a meta tag, it is ONLY a very small part of the algorithm.

If your wanting to use it on your site, go ahead... its not going to hurt your site, UNLESS you 'spam' your key words in the meta tags.
Either way, I would not spend much time working on your meta tags. Spend the most time on your website content.

Content is king on the web!!

Vladd44 · 01-28-2004, 08:56 AM

the primary search engine to be concerned about is google. Its results are used in some form or another on many of the other engines.

As Larry said, meta tags have mininal effect.

Keyword in url is considered. Page title, and text content.

Also page ranking is important, and that is devised by the number of pages that link to your site, traffic and other unknown relevancy factors not revealed.

cloaking, spamming keywords etc is a quick way to get banned from google. And From personal experience , over 1/3 of my traffic comes from google alone.

01-20-2004, 08:07 PM	#1 (permalink)
mac_mogul Monster Techie Join Date: Jul 2003 Posts: 1,295	Search engine question Something I've always wondered about, How exactly does a search engine work? __________________ "Who are you?" "Me? I'm the living embodiment of all that is evil in the computer world." "You're Bill Gates?" iBook G4 1.2ghz PowerPC G4 // 1.25gb PC2700 DDR // ATI RADEON Mobility 9200 32mb DDR // 60gb Ultra ATA/100 Hitachi HDD // Matshita 24x CD-RW/DVD Combo // M-Audio Sonica Theater 7.1 USB Sound Card // Logitech X-530 5.1 Surround

01-21-2004, 08:49 AM	#3 (permalink)
T1junkie Super Techie Join Date: Dec 2003 Posts: 302	search engines have special programs called spiders that are constantly searching around the web for new pages and logging the addresses thats what the meta tags in the HTML soiurce code are for (if you have ever looked) __________________ IBM/Lenovo T43 2687DSU Pentium M 2.0Ghz 1024 MB (DDR2 SDRAM) 80 GB Ultra ATA, 5400 RPM 14.1\" LCD Windows XP Pro DVD-R 64MB ATI MOBILITY RADEON X300 Integrated 802.11A/B/G, Bluetooth, 10/100/1000 Eth.

01-22-2004, 12:57 PM	#4 (permalink)
mac_mogul Monster Techie Join Date: Jul 2003 Posts: 1,295	Hmm, I thought it seemed too easy... And wow larry, that's knowledge at it's best right there! Incredible that you know this kinda stuff! And thanks for the answer, it clears things up alot! For the longest time I thought it was just like some sort of magic-technology thing...ww... __________________ "Who are you?" "Me? I'm the living embodiment of all that is evil in the computer world." "You're Bill Gates?" iBook G4 1.2ghz PowerPC G4 // 1.25gb PC2700 DDR // ATI RADEON Mobility 9200 32mb DDR // 60gb Ultra ATA/100 Hitachi HDD // Matshita 24x CD-RW/DVD Combo // M-Audio Sonica Theater 7.1 USB Sound Card // Logitech X-530 5.1 Surround

01-28-2004, 08:56 AM	#7 (permalink)
Vladd44 Junior Techie Join Date: Jan 2004 Posts: 51	the primary search engine to be concerned about is google. Its results are used in some form or another on many of the other engines. As Larry said, meta tags have mininal effect. Keyword in url is considered. Page title, and text content. Also page ranking is important, and that is devised by the number of pages that link to your site, traffic and other unknown relevancy factors not revealed. cloaking, spamming keywords etc is a quick way to get banned from google. And From personal experience , over 1/3 of my traffic comes from google alone. __________________ For WinMX and BitTorrent help go here

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

01-20-2004, 10:11 PM	#2 (permalink)
Larry Administrator Join Date: May 2003 Posts: 1,410	Lets see if I can sum up the key areas of a search engine: 1) A user enters a key word or key phrase 2) The "search engine" then "returns a result set" of "links" that the search engine "thinks" is "most relevant" to the key word or phrase Now, that was easy, I will now try to explain the meanings of the words in quotes "search engine"- obviously there are the main 3 (google, yahoo, msn), but a search engine is an application that has a simple front-end interface and a HUGE database of words and websites. "returns a result set"- is a list of websites ranked in order based upon the search engine's "algorithm" "algorithm"- each search engine has its own special code or algorithm that sorts or ranks the database full of websites. Needless to say, the algorithm is VERY, VERY complex. Its made of thousands and thousands of variables that determine the sort order for key words entered into the search engine. "links"- basically each link is a website that has been indexed or spidered or inserted into the search engine's database "thinks"- the search engine's brains, if you will, is the algorithm. It has rules and values that help determine the sort order of listing a result set. "most relevant"- again, determined by the algorithm, the search engine's job is to give you a list of websites IT THINKS is what you want. Sometimes it works. Sometimes the search engine fails to return to the user, what the user is looking for. hope that helps!

01-25-2004, 06:52 PM	#5 (permalink)
Vicious Newb Techie Join Date: Jan 2004 Posts: 11	How Internet Search Engines Work http://computer.howstuffworks.com/search-engine.htm

01-26-2004, 02:34 AM	#6 (permalink)
Larry Administrator Join Date: May 2003 Posts: 1,410	Most major search engines do not really use meta-tags, at least in the way you mean. Even if a search engine does use a meta tag, it is ONLY a very small part of the algorithm. If your wanting to use it on your site, go ahead... its not going to hurt your site, UNLESS you 'spam' your key words in the meta tags. Either way, I would not spend much time working on your meta tags. Spend the most time on your website content. Content is king on the web!!