Categories
Uncategorized

Competing and spinoff projects

 

Logo history[edit]

Content[edit]

Gnuhoo borrowed the basic outline for its initial ontology from Usenet. In 1998, Rich Skrenta said, “I took a long list of groups and hand-edited them into a hierarchy.”[19] For example, the topic covered by the comp.ai.alife newsgroup was represented by the category Computers/AI/Artificial_Life. The original divisions were for AdultArtsBusinessComputersGamesHealthHomeNewsRecreationReferenceRegionalScienceShoppingSocietySports and “World”. While these sixteen top-level categories have remained intact, the ontology of second- and lower-level categories has undergone a gradual evolution; significant changes are initiated by discussion among editors and then implemented when consensus has been reached.

In July 1998, the directory became multilingual with the addition of the World top-level category. The remainder of the directory lists only English language sites. By May 2005, seventy-five languages were represented. The growth rate of the non-English components of the directory has been greater than the English component since 2002. While the English component of the directory held almost 75% of the sites in 2003, the World level grew to over 1.5 million sites as of May 2005, forming roughly one-third of the directory. The ontology in non-English categories generally mirrors that of the English directory, although exceptions which reflect language differences are quite common.

Several of the top-level categories have unique characteristics. The Adult category is not present on the directory homepage but it is fully available in the RDF dump that DMOZ provides. While the bulk of the directory is categorized primarily by topic, the Regional category is categorized primarily by region. This has led many to view DMOZ as two parallel directories: Regional and Topical.

On November 14, 2000, a special directory within DMOZ was created for people under 18 years of age.[20] Key factors distinguishing this “Kids and Teens” area from the main directory are:

  • stricter guidelines which limit the listing of sites to those which are targeted or “appropriate” for people under 18 years of age;[21]
  • category names, as well as site descriptions, use vocabulary which is “age-appropriate”;
  • age tags on each listing distinguish content appropriate for kids (age 12 and under), teens (13 to 15 years old), and mature teens (16 to 18 years old);
  • Kids and Teens content is available as a separate RDF dump;
  • editing permissions are such that the community is parallel to that of DMOZ.

By May 2005, this portion of DMOZ included over 32,000 site listings.

Since early 2004, the whole site has been in UTF-8 encoding. Prior to this, the encoding used to be ISO 8859-1 for English language categories and a language-dependent character set for other languages. The RDF dumps have been encoded in UTF-8 since early 2000.

Maintenance[edit]

Directory listings are maintained by editors. While some editors focus on the addition of new listings, others focus on maintaining the existing listings and some do both. This includes tasks such as the editing of individual listings to correct spelling and/or grammatical errors, as well as monitoring the status of linked sites. Still others go through site submissions to remove spam and duplicate submissions.

Robozilla is a Web crawler written to check the status of all sites listed in DMOZ. Periodically, Robozilla will flag sites which appear to have moved or disappeared and editors follow up to check the sites and take action. This process is critical for the directory in striving to achieve one of its founding goals: to reduce the link rot in web directories. Shortly after each run, the sites marked with errors are automatically moved to the unreviewed queue where editors may investigate them when time permits.

Due to the popularity of DMOZ and its resulting impact on search engine rankings (See PageRank), domains with lapsed registration that are listed on DMOZ have attracted domain hijacking, an issue that has been addressed by regularly removing expired domains from the directory.

While corporate funding and staff for DMOZ have diminished in recent years, volunteers have created editing tools such as linkcheckers to supplement Robozilla, category crawlers, spellcheckers, search tools that directly sift a recent RDF dump, bookmarklets to help automate some editing functions, mozilla based add-ons,[22] and tools to help work through unreviewed queues.

Leave a Reply

Your email address will not be published. Required fields are marked *