Sunday, September 27, 2009

Watson - An efficient access point to online ontologies - A gateway to the Semantic Web

Next generation semantic applications will be characterized by a large number of sometimes widely-distributed ontologies, some of them constantly evolving. That is, many next-generation semantic applications will rely on ontologies embedded in a network of already existing ontologies. Other semantic applications – e.g. some electronic health records (EHR) – will maintain a single, globally consistent semantic model that serves the needs of application developers and fully integrates a number of pre-existing ontologies.

As the Semantic Web gains momentum, more and more semantic data is becoming available online. Semantic Web applications need an efficient access point to this Semantic Web data. Watson, the main focus of this post, provides such a gateway. Two limited demonstrations of Watson - one video, the other static - are given below.

Overview of Watson Functionalities

The role of a gateway to the Semantic Web is to provide an efficient access point to online ontologies and semantic data. Therefore, such a gateway plays three main roles:

(1) it collects the available semantic content on the Web
(2) analyzes it to extract useful metadata and indexes, and
(3) implements efficient query facilities to access the data.

Watson provides a variety of access mechanisms, both for human users and software programs. The combination of mechanisms for searching semantic documents (keyword search), retrieving metadata about these documents and querying their content (e.g., through SPARQL) provides all the necessary elements for applications to select and exploit online semantic resources in a lightweight fashion, without having to download the corresponding ontologies.

For a easy-to-follow video demonstration of The Watson plug-in for the NeOn toolkit, click on

http://videolectures.net/iswc07_daquin_watson/

and, better still, click one of the Media Player links at this destination.

Note: There is a Watson plug-in for the ontology editor Protégé in the works.

Protégé (see my August 24 post below) is probably the most popular ontology editor available. In addition, its well established plug-in system facilitates the development of a plug-in using the Waston Web Services and API. To date, however, the Protégé site provides only what it describes as, “more a proof of concept or an example than a real plug-in.”

NeOn Toolkit

The NeOn architechture for ontology management supports the next generation semantics-based applications. The NeOn architecture is designed in an open and modular way and includes infrastructure services such as a registry and a repository and supports distributed components for ontology development, reasoning and collaboration in networked environments.

The NeOn toolkit, the reference implementation of the NeOn architechture, is based on the Eclipse infrastructure.

Ontology Management
Semantic Web, Semantic Web Services, and Business Applications
Copyright 2008 Springer

A static demonstration of the Watson plug-in for the NeOn toolkit

The Watson plug-in allows the user to select entities of the currently edited ontology he/she would like to inspect, and to automatically trigger queries to Watson, as a remote Web service. Results of these queries, i.e. semantic descriptions of the selected entities in online ontologies, are displayed in an additional view allowing further interactions. The figure below provides an example, where the user has selected the concept “human” and triggered a Watson search. The view on the right provides the query results (a list of definitions of class human which have been found on the Semantic Web) and allows easy integration of the results by simply clicking on one of the different “add”-buttons.

Finally, the core of the plug-in is the component that interacts with the core of the NeOn toolkit: its datamodel. Statements retrieved by Watson from external ontologies can be integrated in the edited ontology, requiring for the plug-in to extend this ontology through the NeOn toolkit datamodel and data management component.



{click on the image above for a larger view}

An interesting exercise:
And search on "snomed"


And, this dynamic view is what you get after clicking the (view as graph) link.

Thursday, September 24, 2009

Querying Semantic Data & Ontology - Assisted Querying of Relational Data --- SQL

My September 11 post discussed the i2b2 suite of applications, which has at its base a collection of database tables – with a star schema format - developed from the ground up to represent ontologies. In the present post, I’ll continue this discussion, only for the case where external ontologies are used. I’ll illustrate this latter option with two examples: querying semantic data & ontology-assisted querying of relational data, both using SQL.

Some organizations are using semantic approaches to create an information model (the ontology) based on data schema taken from a particular organization or industry. Individual application database schema are mapped to a standard information model in order to make the meaning of the concepts in different, application-specific data schema explicit and relate them to each other. The resulting information architecture provides a unified view of the data sources in the organization.

As shown in the figure below, application users can query these semantic (metadata) models, which comprise RDF data or ontologies. Standard ontologies reconcile queries needing access to heterogeneous data sources and application-specific schema. This results in solutions that have the power to address problems such as:

* data integration across a heterogeneous, expanding set of sources,
* racking provenance information, and
* modeling probabilistic data and schema.

The product focused on in this post – chosen in part by the toss of a coin – is the latest database from Oracle, 11g, and not competitors like SQL Server. Oracle, it should be mentioned, can deploy on any server platform (Unix, Linux, or Windows) whereas Microsoft SQL Server can deploy only on Windows Server.

In Oracle 11g,
RDF triples based on a graph data model are persistent, indexed, and queried, similar to other object-relational data types. I’ll have more to say on RDF/OWL data and ontologies in future posts. For now, the links found earlier in this paragraph serve as an introduction.


As shown in this figure, the Oracle 11g database contains semantic data and ontologies (RDF/OWL models), as well as traditional relational data.

The Oracle Database 11g semantic database features enable:

* Storage, Loading, and DML access to RDF/OWL data and ontologies
* Inference using OWL and RDFS semantics and also user-defined rules
* Querying of RDF/OWL data and ontologies using SPARQL-like graph patterns
* Ontology-assisted querying of enterprise (relational) data


Query Semantic Data in Oracle Database

RDF/OWL data can be queried using SQL. The Oracle SEM_MATCH table function, which can be embedded in a SQL query, has the ability to search for an arbitrary pattern against the RDF/OWL models, and optionally, data inferred using RDFS, OWL, and user-defined rules. The SEM_MATCH function meets most of the requirements identified by W3C SPARQL standard for graph queries. Support for virtual models, a view-like feature for combining models and optionally corresponding entailments from a UNION or UNION ALL operation, can be used in a SEM_MATCH query. New in release 11.2 of the Oracle database, the SPARQL FILTER, UNION, and OPTIONAL keywords are supported in the SEM_MATCH table function.

{click on the image above for larger view}

Ontology-assisted Query for Relational Data

Queries can extract more semantically complete results from relational data by associating relational data with ontologies that organize the domain knowledge of the relational data.

As shown in the next example, Oracle 11g performs this task by associating an ontology with the data and using the new SEM_RELATED operator (and optionally its SEM_DISTANCE ancillary operator). The new SEM_INDEXTYPE index type improves performance for semantic queries.


{click on the image above for larger view}

For an in-depth treatment of the SEM_MATCH table function, the SEM_RELATED operator, and related topics, consult the Oracle Database Semantic Technologies Developer's Guide.

Native Inferencing using OWL, RDFS, and user-defined rules

In addition to simply storing, and querying an ontology, the latest Oracle database can perform a number of other important tasks, including but not limited to drawing inferences and reasoning. The ability to draw inferences from existing data using the precision and rigor of mathematical logic (e.g., Description Logic) is probably the most important property that distinguishes semantic data from others. New Oracle Database 11g enhancements include a native inference engine for efficient and scalable inferencing using major subsets of OWL. This OWL inferencing engine makes the existing native inferencing for RDF, RDFS, and user-defined rules (used for additional specialized inferencing capabilities) more efficient and scalable. Inferencing may also be done using any combinations of these various entailment regimes. In addition, through the Oracle Jena Adaptor (downloadable from the Oracle Semantic Technologies page), you can integrate with external reasoners such as Pellet (see my August 24 post below for an introduction to Pellet).

Friday, September 11, 2009

Functional Design of an Ontology --- Relationship of the i2b2 ontology to star schema

Recent posts to this blog have discussed ontologies and description logics . As mentioned earlier, the OWL-DL and OWL-Lite sub-languages of the W3C-endorsed Web Ontology Language (OWL) are based on a description logic. A tool for editing and creating ontologies, Protégé, was also described.

In the present post, I’d like to describe the functional design of the ontology used by i2b2, a collection of open-source software tools for the collection and management of project-related clinical research data. That is, this post will present an introduction to what’s under the hood.

Data storage

i2b2 data is stored in a relational database, usually either Oracle or SQL Server and always in a star schema format, a design proposed initially by Ralph Kimball in the 1980s. It is named this because of the appearance of the final database schema diagram that looks like a star (see figure below).

Notes: Ralph Kimball and I were formerly regular contributors to now-defunct DBMS Magazine. A brief introduction to the star schema format is given in the OLAP section of my article Using Neural Networks and OLAP Tools to Make Business Decisions. (See the bibliography at the bottom of this blog)





{click on the images above for larger views}

A star schema contains one fact and many dimension tables. The fact table contains the quantitative or factual data, while the dimension tables contain descriptors that further characterize the facts.


Facts are defined by concept codes and the hierarchical structure of these codes together with their descriptive terms and some other information forms the i2b2 ontology (also called metadata).

i2b2 ontology data may consist of one or many tables. If there is one table, it will contain all the possible data types or categories. The other option is to have one table for each data type. Examples of data types are: diagnoses, procedures, demographics, lab tests, encounters (visits or observations), providers, health history, transfusion data, microbiology data and various types of genetics data. All metadata tables must have the same basic structure.

The structure of the metadata is integral to the visualization of concepts in the i2b2 tools, as well as for querying the data.

In healthcare, a logical fact is an observation on a patient. It is important to note that an observation may not represent the onset or date of the condition or event being described, but instead is simply a recording or a notation of something. For example, the observation of ‘diabetes’ recorded in the database as a ‘fact’ at a particular time does not mean that the condition of diabetes began exactly at that time, only that a diagnosis was recorded at that time (there may be many diagnoses of diabetes for this patient over time).

The fact table contains the basic attributes about the observation, such as the patient and provider numbers, a concept code for the concept observed, a start and end date, and other parameters. In i2b2, as shown in the figure above, the fact table is called observation_fact.

Dimension tables contain further descriptive and analytical information about attributes in the fact table. A dimension table may contain information about how certain data is organized, such as a hierarchy that can be used to categorize or summarize the data. In the i2b2 Data Mart, there are four dimension tables that provide additional information about fields in the fact table: patient_dimension, concept_dimension, visit_dimension, and provider_dimension.

Scalability

Once a database grows to over 10 million items, the advantages of a star schema can start to take hold. The first consideration is the speed and integrity of the queries. When one exceeds 0.5 billion rows in a database, it becomes important to have the data expressed in very large indexes. Very large indexes are only possible with very large tables. If one has several hundred or thousand tables in a database (easily attained in large transaction systems), one will have at least one index on each table resulting in several hundred or thousand small indexes. Joins between 100‐1000 indexes for each query will result in slow performance (hours), while joins between 3‐4 indexes, even representing 100’s of millions of rows, will be fast (seconds). Furthermore, the integrity of queries in a transactional database is also compromised because queries can often be answered through several paths in a circular manner.

The second consideration is the need for a large analytic database to constantly absorb new data. The database schema does not change as new data sources are added. New data will result in additional rows added to the fact, patient, and visit tables. New concepts and observers will result in new rows added to the concept and provider tables. But new columns and tables do not need to be added for each new data source. This is very useful in large projects where there are many tools depending upon a specific database schema. A strategy where the database grows by adding rows for new data rather than adding new tables and columns allows tools developed to work with one kind of data to also work with a new source of data.

The third advantage of the star schema is the ability to manage the metadata of a large analytic database. Metadata is used to perform queries, and if it is incorrect a query will be profoundly affected. For example, if one wanted to find all the patients with diabetes, but left out one of the codes used to represent diabetes in a database, none of those orphaned patients would be counted. The detection of orphaned concepts is easily achieved in the star schema by, for example, joining the fact table to the concept and provider tables and reporting those fact table concepts and providers left out by the join.

A sample ontology query for diagnosis

To find all the patients that were diagnosed with migraines, use this query:

Select distinct (patient_num)
From observation_fact
Where concept_cd in
(select concept_cd
from concept_dimension
where concept_path like '%Neurologic Disorders (320-389)\(346) Migraine\%')

Note: The material in this post has been taken largely from the following i2b2 pages, which should be consulted for further details:

https://www.i2b2.org/software/projects/ontologymgmt/Ontology_Design_Doc_13.pdf

https://www.i2b2.org/software/projects/workbench/i2b2Workbench_Description.pdf

https://www.i2b2.org/software/projects/datarepo/CRC_Design_Doc_13.pdf




Wednesday, September 9, 2009

Semantic Interoperability, EHR, etc. are of little use to someone whose claim is denied by his or her insurance company

Founded in 1945, Kaiser Permanente is this nation’s largest not-for-profit health plan, serving more than 8.6 million members, with headquarters in Oakland, California.

This blog has carried a prominently-placed electronic health records (EHR) video outlining some of the excellent information technology work that's being introduced by Kaiser Permanente.

So, given the placement of this video, I feel a responsibility to add here the reality that Kaiser Permanente's technology is only one facet of a system that daily makes decisions about who can and who cannot get health care.

The California Nurses Association/National Nurses Organizing Committee has just released new data that reveals more than one of every five requests for medical claims for insured patients, even when recommended by a patient’s physician, are rejected by California’s largest private insurers. (The Kaiser Permanente Health Plan membership in California is greater than 6 million.)

This is data that the health insurance companies have wanted to hide, and it’s just now becoming available. It documents that these insurance companies have denied, in California alone, 45 million claims since 2002. Some of these rates ranged as high as 40 percent (for UnitedHealthcare’s PacifiCare). And other large, giant insurers like Blue Cross, Health Net, CIGNA, and Kaiser were all in the range of 30 percent (Kaiser Permanente's denial rates is 28 percent). This report shows a clear pattern of very high denials by the very insurance companies that people depend upon to assure that they get the care they need when they need it.

There are a variety of reasons insurance companies claim why they make these denials: in the end though, it’s a war that goes on between the insurance companies and the doctors and the hospitals. (Note: Attorney General of California Jerry Brown has announced he’s going to conduct an investigation into the business practices of these companies and why these denial rates are so high.)

A recent piece in the Los Angeles Times quotes a spokeswoman for the California Association of Health Plans, responding to the data that the California Nurses Association/National Nurses Organizing Committee has just released, saying, “It appears [that] a good deal of the so-called denials are merely paperwork issues.”

It seems to me that even if you put the best face on the California Association of Health Plans' response, what it demonstrates is how much waste (aka administrative overhead) there is in the health insurance industry. It's been suspected for some time now that one-third of every healthcare dollar goes to waste and to enforcing claims denials in the United States..

Monday, August 24, 2009

Reasoning for Ontology Engineering and Usage and The Challenges of Modern Medical Ontologies

In my August 6 post, I briefly introduced ontology editor Protégé 4.0 with the reasoners FaCT++ (implemented using C++) and Pellet (Java based). Today's post picks up this story and adds the RacerPro (commercial) reasoner to the mix. You can -- and I recommend that you do -- download your own copies of the latest versions of these tools. Links that enable you to do so are located at the end of this post.



Protégé 4.0 with three reasoner added-ins

A
reasoner is a piece of software able to infer logical consequences from a set of asserted facts or axioms. In the present context, a reasoner makes inferences about classes and individuals in an ontology, tasks that are beyond the Web Ontology Language (OWL) model alone.

Ontologies, as described in prior posts, are formal vocabularies of terms, often shared by a community of users, and, as such, ontologies play an important role in semantic interoperability and Web 3.0. One of the most prominent application areas of ontologies is medicine and the life sciences. For example, the Systematised Nomenclature of Medicine Clinical Terms (SNOMED CT) is a clinical ontology. Another example is the OBO Foundry -- a repository containing about 80 biomedical ontologies.

These ontologies are gradually superseding existing medical classifications and will provide the future platforms for gathering and sharing medical knowledge. Capturing medical records using ontologies will reduce the possibility for data misinterpretation, and will enable information exchange between different applications and institutions.

Medical ontologies are strongly related to description logics (DLs), which provide the formal basis for many ontology languages, most notably the W3C standardised OWL. All the above mentioned ontologies are nowadays available in OWL and, therefore, in a description logic. The developers of medical ontologies have recognised the numerous benefits of using DLs, such as the clear and unambiguous semantics for different modelling constructs, the well-understood tradeoffs between expressivity and computational complexity, and the availability of provably correct reasoners and tools (discussion to follow).

The development and application of ontologies crucially depend on reasoning. Ontology classification, i.e., organising classes into a specialisation/generalisation hierarchy, is a reasoning task that plays a major role during ontology development: it provides for the detection of potential modelling errors such as inconsistent class descriptions and missing sub-class relationships. For example, about 180 missing sub-class relationships were detected when the version of SNOMED CT used by the NHS was classified using the DL reasoner FaCT++. Query answering is another reasoning task that is mainly used during ontology-based information retrieval; e.g., in clinical applications query answering might be used to retrieve "all patients that suffer from nut allergies".

Despite the impressive state-of-the-art, modern medical ontologies pose significant challenges to both the theory and practice of DL-based languages. Existing reasoners can efficiently deal with some large ontologies, but many important ontologies are still beyond the reach of available tools (i.e., they are unable to classify some widely used ontologies).

Applications currently need to work around these limitations, e.g., by using subsets of ontologies that can be successfully processed. For example, the version of
GALEN typically used in practice contains only about 20% of the axioms of the full version; this reduces the interaction between concepts and thus makes the ontology "processable". This is, however, highly undesirable in practice, because it reduces coverage, weakens the conceptualisation of the domain and may prevent the detection of modelling errors.

Furthermore, the amount of data used with ontologies can be orders of magnitude larger than the ontology itself. For example, the annotation of patients' medical records in a single hospital can easily produce data consisting of hundreds of millions of facts, and aggregation at a national level might produce billions of facts. Existing reasoners cannot cope with such data volumes, especially not if ontologies such as
GALEN and FMA are used as schemata.

Having forewarned you about these limitations, I'd like to recommend the following video on reasoners -- free and commercial.

http://videolectures.net/iswc08_moller_itsr/

Some readers of this blog might not be familiar with terms that appear in the video, starting with ABox and TBox.



For those readers especially, the following links for accessing the Protégé, Pellet and Racer sites could be used to install and examine this software (and accompaning documentation) before watching the videos.

Protégé http://protege.stanford.edu/

Pellet http://clarkparsia.com/pellet/

RacerPro http://www.racer-systems.com/

Then, as you watch the video, you could follow along using your own running code. For some, this will require more than a single session.

Recommended reading - basics of description logics:
www.inf.unibz.it/~franconi/dl/course/dlhb/dlhb-01.pdf
www.inf.unibz.it/~franconi/dl/course/dlhb/dlhb-02.pdf
http://www.cs.man.ac.uk/~horrocks/Slides/IJCAR-tutorial/Print/p1-introduction.pdf

Thursday, August 13, 2009

Semantic Web Technologies: Ontologies, Agents, Web Services


Here's a Web app developed at Stanford University some time ago. It's made up of a single ontology with entries for foods and wines, an ontology agent that returns text from this ontology and a portal that manipulates the text returned by the agent.

http://onto.stanford.edu:8080/wino/index.jsp

The ontology underlying the agent used here contains hierarchies and descriptions of food and wine categories, along with restrictions on how particular instances might be paired together.

For readers wanting an easy-to-read discussion on what exactly an agent is and how it works with ontologies, see

http://www.nature.com/nature/webmatters/agents/agents.html

The next three figures are of screen shots taken from this Web app, which doesn't always employ state-of-the-art technologies like OWL (discussed in earlier posts) under the hood; but, this app provides a simple example of the most basic building blocks that are found in many of today's Semantic Web apps (also discussed in earlier posts).



Figure I - The home page




Figure II - After clicking on the "Roast Duck" link




Figure III - After clicking on the "Web Inventory Search" link

The ontology defines the concept of a wine. According to the specification, a wine is a potable liquid produced by at least one maker of type winery, and is made from at least one type of grape (such grapes are restricted to wine grapes elsewhere in the ontology.)

The declaration additionally stipulates that a wine comes from a region that is wine-producing and, most importantly to the agent, that a wine has four properties: color, sugar, body, and flavor.

The concept of a meal course underlies pairing of a food with a wine. Each course is a consumable thing comprising at least one food and at least one drink, the latter of which is stipulated to be a wine.

When the user selects a type of course, or an individual food that gets mapped to a type of course, the agent will consult that course definition for restrictions on the constituent food or wine. All such course types map back to this concept, like objects to their superclasses in object-oriented programming.

Suppose the user has selected pasta with fra diavolo, or perhaps pasta with spicy red sauce directly. The concept of such a food is defined elsewhere in the ontology. Furthermore, such courses must be a subclass of those with specific restrictions on the properties of their wines.

One wine that matches the above restrictions is the Pauillac. This individual wine is simply defined as a Pauillac whose maker is Chateau Lafite Rothschild. Together with other statements in the ontology, this allows the reasoner (discussed in an earlier post) to deduce many additional facts: that this a Medoc wine from Bordeaux, in France, and that it is red, to name a few.

The concept of a Pauillac specifies that all such wines feature full bodies and strong flavors and are made entirely from cabernet sauvignon grapes. Further, Pauillacs are a particular subset of Medocs, distinguished by their origin in the Pauillac region. It is through this additional subclass relationship that Pauillac are defined elsewhere as red and dry.

Following the above example through the ontology reveals a straightforward logical path for pairing the Pauillac with the selected course. Because these items were specified in a standardized, machine-readable format, it is an equally straightforward task for any compliant automated reasoner.

Why use an ontology?

The functionality provided by the wine agent is not unlike that which could be provided by a simple look-up table. Indeed, food/wine pairings are traditionally published in some form of tabular chart where marks appear at the intersections of columns and rows representing compatible varieties of food and wine. The wine agent demonstrates that at least this simple task can be accomplished within semantic markup technology, but what about more complicated applications like, for example, those required by electronic health rcords (EHR)? For now, I'll postpone any discussion of EHR and stick with this food-wine pairing app.

Suppose that if not the entire web, then at least some number of cooperating parties were using semantic markup to participate in this project. Rather than the traditional approach of trying to build an enormous database of foods and wines, then, the definitions would be distributed across the participating parties. A restaurant or retailer offering an on-line menu could mark each food item with standardized machine-readable definitions. Similarly, then, a wine retailer could mark its wines according to the definitions exemplified above.

Such markings would benefit from well-known advantages of ontologies. For instance, through the subclassing, adding a new Pauillac to the inventory would not require wine.com (the source of the GUI shown in Figure III) to mark all of the wine's properties; it would just be another Pauillac as specified in the example, plus any differentiating features. But more importantly in terms of software agents, all the markings would be machine readable, and could be handled by systems from any organization.

Rather than relying on a human user to select a food or food type, the agent could crawl the web for foods marked within the wines namespace and pre-compile suitable pairings. This is where Web services (discussed in earlier posts) come in.

Footnote

The notion of agent-based computing has been adopted enthusiastically in the financial trading community, where autonomous market trading agents have been said to outperform human commodity traders by 7%.

Machines can monitor stock market movements much more quickly than humans, and if you can encode the kinds of rules that you want, then it is not unreasonable to imagine that computational traders will be able to outperform humans.

I commented on high-speed trading in my August 3 post.

Monday, August 10, 2009

SNOMED, OWL, Ontologies, Protege, etc. Hardware and Software Issues.

In my last two posts, I rather glibly talked about SNOMED, OWL, ontologies, Protege, etc. without mentioning any hardware or software issues that sometimes arise.

For most of us, these matters are best understood from our own "trial and error" experiences. Nonetheless, I've added links to a couple of informative anecdotes below: in one, the account of someone computing successfully with 8 dual CPUs and 15 gig RAM; in another, the account of someone computing unsuccessfully having far less resources.

Link 1: Modeling Massive Ontologies (SNOMED) at Kaiser

Link 2: SNOMED OWL On Protege

Footnote 1: Some large OWL files will not load into Protege on a 32-bit computer when to do so requires in excess of 4 Gigs RAM, the theoretical limit of a 32-bit system. In order to get around this limitation, a 64 bit machine is needed. Fortunately, the Linux, Windows, and Mac operating systems are available in 64-bit versions.

Footnote 2: The entire computing industry is moving from 32-bit to 64-bit technology, and it’s easy to see why. While many of today’s computers can hold far more than 4GB of physical memory, the 32-bit applications that run on them can address only 4GB of RAM at a time. 64-bit computing shatters that barrier by enabling applications to address a theoretical 16 billion gigabytes of memory, or 16 exabytes. 64-bit machines can also crunch twice the data per clock cycle, which can dramatically speed up numeric calculations and other tasks.

Friday, August 7, 2009

Ontologies, Ontology Languages, and Semantic Interoperability -- Segue to Electronic Health Records (EHR)


This post serves as a transition between my previous one on OWL and ontologies and my upcoming ones on electronic health records (EHR) - past, present and future.

OWL is the latest standard in ontology languages from the World Wide Web Consortium (W3C) - it is built on top of RDF (i.e., OWL semantically extends RDF).

These two languages are explained in

http://www.co-ode.org/resources/tutorials/intro/slides/OWLFoundationsSlides.pdf

As an example of an ontology that's central to building an interoperable EHR system, I'll cite the Systematized Nomenclature of Medicine (SNOMED) ontology, which is used in more than 50 countries around the world:



{click to enlarge}

The Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) Ontology includes a Core terminology of over 364,000 health care concepts with unique meanings and formal logic—based definitions organized into hierarchies. As of January 2005, the fully populated table with unique descriptions for each concept contained more than 984,000 descriptions. Approximately 1.45 million semantic relationships exist to enable reliability and consistency of data retrieval. SNOMED CT is available in English, Spanish and German language editions.

What is its structure?

Core content includes the concepts table, descriptions table, relationships table, history table, an ICD-9-CM mapping, and the Technical Reference Guide. And, it can map to other medical terminologies and classification systems already in use.

SNOMED is meant to be complementary to LOINC (Logical Observations Identifiers, Names, Codes), another clinical terminology important for laboratory test orders and results.

See http://www.ihtsdo.org/ for a good deal more information on SNOMED.

For advanced IT readers

The DL that SNOMED uses is much less expressive than OWL. The result is, even though you can mechanically translate SNOMED into OWL, the resulting OWL ontology will be very unlike anything an OWL author would create starting from scratch, and might also be a challenge to classify successfully under an OWL reasoner without a lot of manual editing.

Furthermore, even at that point it would also be of limited value in supporting reasoning over OWL instances, as many kinds of assertions that would be routine in an OWL ontology (like disjoints, explicit domain/range constraints, etc.) do not exist in native SNOMED and would have to be created.

One opinion holds that creating your own OWL ontology and using SNOMED as a mapping target leverages SNOMED in a more useful way for most conceivable applications.

Finally, for a good deal more on the topics covered so far, consider

Clinical Decision Support Systems
Theory and Practice
Series: Health Informatics
Berner, Eta S. (Ed.)
2nd ed., 2007
ISBN: 978-0-387-33914-6

Thursday, August 6, 2009

Semantic Interoperability -- Part III Ontologies -- Prelude to Electronic Health Records (EHR)

This post is a continuation of the introduction to ontologies that I posted on August 4 and July 20.

Readers new to this subject might also find the 2 ½ minute video What Is Web 3.0, Anyway? worthwhile.

An ontology is an explicit specification of a conceptualization (defined earlier), that is to say, a formal representation of a knowledge domain. Usually an ontology consists of: (i) classes, which represent the concepts of the domain (for example, in an ontology about the domain of Telecommunications, as in the listing below, a possible concept could be "Phone"); (ii) properties, to establish relationships between the concepts (for example, a "Phone" concept could have as property the "Company"; (iii) instances, with concrete examples associated with every concept (for example, "Siemens" could be an instance of the "Company” concept); and (iv) axioms, which are restrictions applicable to certain elements of the ontology, necessary to specify completely the knowledge domain (for example, in the ontology about telecommunications, it could define a restriction to indicate that in this domain a "Phone" must have always, at least, a "Company").

Ontologies can be stored using XML-based markup languages such as OWL (Ontology Web Language), which facilitates their reuse in different semantic platforms to annotate and search resources. These languages allow us to define tags in order to represent the different ontology elements. The listing below shows an extract of a OWL file containing the Telecommunications example ontology that has been created using the Protegé tool. As you can observe, in this language, the concepts are delimited by the Class tag, the properties by the ObjectProperty tag, the instances by the tag corresponding to the associate class (in the example, the class Company has as instance "Siemens"), and the axioms with tags like Restriction or subClassOf (this one is used in the example for representing that "Cellphone" is a type of "Phone").



Content of a OWL file - a fragment of an
Ontology about Telecommunications
{click to enlarge}

Today one of the main uses of ontologies is to support the Semantic Web (aka Web 3.0), especially for annotating Web resources and facilitating the localization of these annotated resources when users formulate queries to semantic search engines. For this purpose, in the previous example of Telecommunications, an ontology has included two annotations as instances of the “Phone” and “Cellphone” classes which correspond to two documents (“Gigaset3015Classic.pdf” and “MobileC55.pdf”, respectively) located in a hypothetical Web server (“http://www.telecosiemens.com”).

The Reality

Researchers have written much about the potential benefits of using ontologies, and most of us regard them as central building blocks of the Semantic Web and other semantic systems. Unfortunately, the number and quality of actual, “non-toy” ontologies available on the Web today is remarkably low. This implies that the Semantic Web community has yet to build practically useful ontologies for a lot of relevant domains in order to make the Semantic Web a reality.

In striking contrast to the data within a stand-alone document, publications have yet to benefit from the opportunities offered by cyber infrastructure. While the means of distributing publications has vastly improved, publishers have done little else to capitalize on the electronic medium. In particular, semantic information describing the content of these publications is generally sorely lacking, as is the integration of this information with data in public repositories.

The Reasoner

One of the key features of ontologies is that they can be processed by a reasoner. One of the main services offered by a reasoner is to test whether or not one class is a subclass of another class. By performing such tests on all of the classes in an ontology it is possible for a reasoner to compute the inferred ontology class hierarchy. Another standard service that is offered by reasoners is consistency checking. Based on the description (conditions) of a class, the reasoner can check whether or not it is possible for the class to have any instances. A class is deemed to be inconsistent if it cannot possibly have any instances.

Reasoning with Protégé 4.0

Reasoning with your ontology is one of the most commonly performed activities and the ontology editor Protege 4.0 comes with 2 built-in reasoners, FaCT++ and Pellet. To classify your ontology, open the Reasoner menu and select one of the available reasoners. FaCT++ will automatically classify your ontology. Pellet requires that you select classify. Once you have done this, the class hierarchy on the Entites tab changes to show the inferred class hierarchy. Unsatisfiable classes appear in red under Nothing, and everything else appears in the hierarchy under their inferred superclasses. The asserted class hierarchy is still available, stacked under the asserted one, as shown in the next screenshot.



Screenshot of an inferred class hierarchy
{click to enlarge}

Instructions for getting started with the OWL editor in Protege 4:
http://protegewiki.stanford.edu/index.php/Protege4GettingStarted

A Practical Guide To Building OWL Ontologies:
http://www.co-ode.org/resources/tutorials/ProtegeOWLTutorial-p4.0.pdf

The Microsoft Word Add-in For Ontology Recognition – An Introduction

There are other tools (e.g., The Microsoft Word Add-in For Ontology Recognition to name one) that might be suitable for your needs. It’s a MS Word 2007 add-in that enables the annotation of Word documents based on terms that appear in ontologies.

This Word Add-in For Ontology Recognition is a free Microsoft download. With it, as shown in the figures below, you select and then download one or more ontologies which are thereafter available automatically from within your Word document.

The Microsoft Word Add-in For Ontology Recognition – An Overview

This add-in enables authors who use Microsoft Word for content creation to incorporate semantic knowledge into the content. This add-in should simplify the development and validation of ontologies, by making ontologies more accessible to a wide audience of authors and by enabling semantic content to be integrated in the authoring experience, capturing the author’s intent and knowledge at the source, and facilitating downstream discoverability.

The goal of the add-in is to assist authors in writing a manuscript that is easily integrated with existing and pending electronic resources. The major aims of this project are to add semantic information as XML mark-up to the manuscript using ontologies and controlled vocabularies (from the National Center for Biomedical Ontology) and identifiers from major biological databases, and to integrate manuscript content with existing public data repositories.

As part of the publishing workflow and archiving process, the terms added by the add-in, providing the semantic information, can be extracted from Word files, as they are stored as custom XML tags as part of the content. The semantic knowledge can then be preserved as the documented is converted to other formats, such as HTML or the XML format from the National Library of Medicine, which is commonly used for archiving.

The full benefit of semantic-rich content will result from an end-to-end approach to the preservation of semantics and metadata through the publishing pipeline, starting with capturing knowledge from the subject experts, the authors, and enabling this knowledge to be preserved when published, as well as made available to search engines and presented to people consuming the content.

The Microsoft Word Add-in For Ontology Recognition – Screen Shots

{click to enlarge}

The Word Add-in For Ontology Recognition User’s Guide for the Semantic Mark-up and XML Formatting of Scholarly Articles is a good place to start for further information on this tool.

Semantic Tagging

When a word or set of words is tagged by the add-in, the word is wrapped with some tags that associate it with the ontology term. The example below shows the word "disease" being tagged with Human Disease ontology.

{click to enlarge}

If the Word file (docx) is to be transformed to other formats, this set of tags would need to be processed using xslt or other technologies. Note that there are other CodePlex projects available which implement transformations of docx files to other formats, which one can start from.

Ontology Add-in for Microsoft Office Word 2007 Video

Tuesday, August 4, 2009

Electronic Health Records (EHR) – Semantic Interoperability – Part 2


Before discussing the connection(s) between electronic health records (EHR) and semantic interoperability directly, I’d like to spend a little time talking about Semantic Web interoperability (not necessarily the same thing as the interoperability of present and future EHS systems).

In general, semantics is the study of meaning. Semantic Web (also called Web 3.0) technologies help separate meanings from data, document content, or application code, using technologies based on open standards. If a computer understands the semantics of a document, it doesn't just interpret the series of characters that make up that document: it understands the document's meaning. See my July 20 post for a brief introduction to this material.



Benefits of the Semantic Web to the World Wide Web

The World Wide Web is the biggest repository of information ever created, with growing contents in various languages and fields of knowledge. Search engines might help you find content containing specific words, but that content might not be exactly what you want. What is lacking? The search is based on the contents of pages and not the semantic meaning of the page's contents or information about the page.

Once the Semantic Web exists, it can provide the ability to tag all content on the Web, describe what each piece of information is about and give semantic meaning to the content item. Thus, search engines become more effective than they are now, and users can find the precise information they are hunting. Organizations that provide various services can tag those services with meaning (service-oriented architectures -- SOA -- are discussed in my June 20 post and mentioned again below, this time in the context of semantics); using Web-based software agents, you can dynamically find these services on the fly and use them to your benefit or in collaboration with other services (See
http://www.oracle.com/technology/pub/articles/matjaz_bpel1.html for a discussion of the orchestration and choreography of Web services).

Ontologies

The use of words to refer to concepts (the meanings of the words used) is very sensitive to the context and the purpose of these words.

An ontology is a formal representation of a set of concepts within a domain and the relationships between those concepts. It may be used to reason about the properties of that domain, and may be used to define the domain. The word “reason” is an important part of this story and will come up again in a later post, when I talk about Protégé, a free, open source ontology editor and knowledge-base framework.

A domain ontology (or domain-specific ontology) models a specific domain, or part of the world (e.g., healthcare, banking or politics). It represents the particular meanings of terms as they apply to that domain. For example the word card has many different meanings. An ontology about the domain of poker would model the "playing card" meaning of the word, while an ontology about the domain of computer hardware would model the "video card" meaning.

For each domain of human knowledge, an ontology must be constructed, partly by hand and partly with the aid of dialog-driven ontology construction tools (to be discussed in an upcoming post).

Ontologies are not knowledge nor are they information. They are meta-information. In other words, ontologies are information about information. In the context of the Semantic Web, they encode, using an ontology language (to be discussed in an upcoming post), the relationships between the various terms within the information. Those relationships, which may be thought of as the axioms (basic assumptions), together with the rules governing the inference process, both enable as well as constrain the interpretation (and well-formed use) of those terms by the Info Agents (to be discussed in an upcoming post) to reason new conclusions based on existing information, i.e. to think. In other words, theorems (formal deductive propositions that are provable based on the axioms and the rules of inference) may be generated by the software, thus allowing formal deductive reasoning at the machine level. And given that an ontology, as described here, is a statement of Logic Theory, two or more independent Info Agents processing the same domain-specific ontology will be able to collaborate and deduce an answer to a query, without being driven by the same software.

When an organization adapts an ontology-driven approach, it can capture and represent its total knowledge in a language-neutral form and deploy the knowledge in a central repository that provides the same semantic meaning across applications.

Semantics are the future of service-oriented integration

To properly model and manage a service-oriented architecture (SOA), enterprise architects must maintain active representations of the services available to the enterprise. Specifically, to discover and organize their services, the architects must use best practices that model and assemble services using metadata, encapsulate business logic in metadata for dynamic binding, and manage with metadata. Ontologies provide a very powerful and flexible way to aggregate, visualize, and normalize this service metadata layer.

Note: When delivering services as part of a service inventory, there is a constant risk that services will be created with overlapping functional boundaries, making it difficult to enable wide-spread reuse. Normalization addresses this problem.
When services are delivered with complementary and well-aligned boundaries, normalization across the inventory is attained. Note also how the quantity of required services is reduced.

Semantic technologies provide an abstraction layer above existing IT technologies, one that enables the bridging and interconnection of data, content, and processes across business and IT silos.

For advanced IT readers

For a rather technical discussions of this topic, see the video Artificial Neural Network based Techniques for Semantic Data Interoperability below and parts of my article Using Neural Networks and OLAP Tools to Make Business Decisions to which there is a link in the bibliography at the very bottom of this blog.





To be continued …



Monday, August 3, 2009

The game is sometimes rigged in favor of the house

High-speed trading: some institutions, including Goldman Sachs, have been using superfast computers to get the jump on other investors, buying or selling stocks a tiny fraction of a second before anyone else can react. Profits from high-frequency trading are one reason Goldman is earning record profits and likely to pay record bonuses.

And there’s a good case that such activities are actually harmful. For example, high-frequency trading probably degrades the stock market’s function, because it’s a kind of tax on investors who lack access to those superfast computers — which means that the money Goldman spends on those computers has a negative effect on national wealth.

So, what's this got to do with information technology in healthcare? Possible, nothing. Remember, however, that one of the stated goals of EHR is to deliver better healthcare at lower cost.


I believe that we should keep in mind that technology - not high-speed computers per se - doesn't always benefit society. As I mention - without much elaboration - in my July 22 post, "EHR / EMR has the potential to facilitate the execution of today's healthcare scams. So, for the time being, we might well have a reason to temper our enthusiasm for the inevitable computerization of our still-largely-paper-bound healthcare records systems."

Saturday, August 1, 2009

Metcalfe's law vis-à-vis the value of semantic interoperability

A new economic model

Metcalfe's law states that the value of a network is proportional to the square of the number of connected users of the system (n**2).

Metcalfe's law characterizes many of the network effects of communication technologies and networks such as the Internet, social networking, and the World Wide Web. It is related to the fact that the number of unique connections in a network of a number of nodes (n) can be expressed mathematically as the number n(n − 1)/2, which is proportional to n**2 asymptotically.



The law has often been illustrated using the example of fax machines: a single fax machine is useless, but the value of every fax machine increases with the total number of fax machines in the network, because the total number of people with whom each user may send and receive documents increases.

Metcalfe's law is more of a heuristic or metaphor than an iron-clad empirical rule. In addition to the difficulty of quantifying the "value" of a network, the mathematical justification measures only the potential number of contacts, i.e., the technological side of a network. However the social utility of a network depends upon the number of nodes in contact. For instance, if Chinese and non-Chinese users don't understand each other, the utility of a network of users that speak the other language is near zero, and the law has to be calculated for the two sub-networks separately.

When considering electronic health record (EHR) interoperability, two nodes are in contact in a meaningful way if the nodes themselves, not just human beings sitting at those nodes, can understand the content of a message from the other node in an unambiguous way. In other words, semantic interoperability is what counts.

Apropos of EHR, Rod Beckstrom, the recently appointed president of the Internet Corporation for Assigned Names and Numbers (Icann), has used his address to the Black Hat USA 2009 conference to propose a new economic model for valuing computer networks and the internet.

"Who cares how many nodes there are?" Beckstrom said. "If you look at a value of the network, focus on the transactions. The value of the network equals the net value added to each user's transactions, summed for all users."

For example, some networks grow the number of users but become less valuable since the value of their transactions is so small.

Microsoft chairman Bill Gates dropping his Facebook account in July was a case in point, Beckstrom said. The number of 'friends' became so great that the network lost its value.

Wednesday, July 22, 2009

Electonic Health Records / Electronic Medical Records - Scams


Dr. David Scheiner, an internist based in the Chicago neighborhood of Hyde Park, was President Obama’s doctor for twenty-two years, from 1987 until he entered the White House but has publicly opposed Obama’s health plan, calling for a single payer system (see the text and video in my May 13, 2009 post for an introduction to the current debate over single payer).

In a recent interview, Dr. Scheiner said, "President Obama believes that when we have electronic records, somehow night will change into day. That won’t happen. First of all, it’s extremely costly. It will become even easier to scam the health insurance companies and Medicare when you have a cursor that can go over all the things that you perhaps didn’t do, but look good on paper, and you can code much higher."

I post this quote simply to remind us all that while electronic health records / electronic medical records will automate and, in all likelihood, improve many aspects of the delivery of healthcare, at the same time EHR / EMR has the potential to facilitate the execution of today's healthcare scams. So, for the time being, we might well have a reason to temper our enthusiasm for the inevitable computerization of our still-largely-paper-bound healthcare records systems.

Monday, July 20, 2009

Electronic Health Records (EHR) - Semantic Interoperability - Part 1


To press a suit means one thing to a tailor and another thing to a lawyer.

A free radical means one thing to a chemist but meant another thing to members of the House Un-American Activities Committee (HUAC) during the 1950’s.

And, medication for pain and pain medication don’t always mean the same thing. The controversy surrounding the recent death of Michael Jackson illustrates this last point.

In the examples above, as in clinical terminology, words can take on different meanings depending on factors like time or place (i.e., context).

Furthermore, clinicians and organizations use different clinical terms that mean the same thing. For example, the terms heart attack, myocardial infarction, and MI may mean the same thing to a cardiologist, but, to a computer, they are all different. There is a need to exchange clinical information consistently between different health care providers, care settings, researchers and others (semantic interoperability), and because medical information is recorded differently from place to place (on paper or electronically), a comprehensive, unified medical terminology system is needed as part of the information infrastructure.

Interoperability

Interoperability is the ability of two parties, either human or machine, to exchange data or information.

First, syntactic interoperability guarantees the exchange of the structure of the data, but carries no assurance that the meaning will be interpreted identically by all parties. Web pages built with HTML or XML are good examples of machine-to-machine syntactic interoperability because a properly structured page can be read by any machine with a Web browser. The meaning of the page to a particular machine may vary substantially; however, this is not usually considered a problem because the semantics of a page are meant to be interpreted by human viewers.

Next, human or semantic interoperability guarantees that the meaning of a structure is unambiguously exchanged between humans. Documents such as progress notes, referrals, consults, and others achieve semantic interoperability at a clinician-to-clinician level by relying on common medical vocabularies.

Finally, computable semantic interoperability requires that the meaning of data be unambiguously exchanged from machine to machine (as shown in the figure below). This does not necessarily mean that all machines need to process the received data the same way, but rather that each machine will make its processing decisions based on the same meaning.



Words and Meanings

The meanings of words change, sometimes rapidly. But a formal language such as used in an ontology -- a rigorous and exhaustive organization of some knowledge domain that is usually hierarchical and contains all the relevant entities and their relations -- can encode the meanings (semantics) of concepts in a form that does not change. In order to determine what is the meaning of a particular word (or term in a database, for example), it is necessary to label each fixed concept representation in an ontology with the word(s) or term(s) that may refer to that concept.

When multiple words refer to the same (fixed) concept, in language this is called synonymy; when one word is used to refer to more than one concept, that is called ambiguity. Ambiguity and synonymy are among the factors that make computer understanding of language very difficult. The use of words to refer to concepts (the meanings of the words used) is very sensitive to the context and the purpose of any use for many human-readable terms.

The use of ontologies in supporting semantic interoperability is to provide a fixed set of concepts whose meanings and relations are stable and can be agreed to by users. When a word used in some interoperability context changes its meaning, then to preserve interoperability it is necessary to change the pointer to the ontology element(s) that specifies the meaning of that word.

There are a number of tools for the programmatic handling (i.e., creating, querying, etc.) of ontologies. The visual representation of ontologies is an important contribution of these tools.



IBM Integrated ontology Development Toolkit (formerly named IBM Semantics Toolkit) is one of many toolkits designed for storage, manipulation, query, and inference of ontologies and corresponding instances.

An upcoming post will discuss the role and value of semantic technology in service-oriented architectures (SOA).

Friday, July 17, 2009

Electronic Health Record Interoperability: Specifications, Standards, and Working Groups

{Prelude to upcoming post - semantic interoperability}

The American Recovery and Reinvestment Act of 2009 (ARRA) states that the
HIT Policy Committee shall make recommendations on standards and implementation specifications, among other tasks.

The
Healthcare Information Technology Standards Panel (HITSP) EHR-Centric Interoperability Specification consolidates all information exchanges that involve an Electronic Health Record (EHR) System within any of the thirteen HITSP Interoperability Specifications existing as of February 13, 2009, the enactment date of the American Recovery and Reinvestment Act (ARRA).

Reading these two statements and others like them requires an understanding of terms such as

*
recommendation
* specification
*
standard, and
*
working group

Each of these bullets is linked to its Wikipedia entry.



Given that many of the readers of this blog (from 42 countries so far) are interested in the information technology aspects of The American Recovery and Reinvestment Act , electronic health record interoperability and the like, it may be easier for them if I elaborate on the four terms listed above by using the example of cascading style sheets (CSS), a simple language that allows you to declare how documents are displayed by Web browsers. By so doing, I’ll be introducing additional expressions with which these members of the IT community are already familiar.

The Cascading Style Sheets language was created through a collaborative effort between Web developers and browser programmers under the auspices of the
World Wide Web Consortium (W3C for short).

The W3C is an international industry group that comprises over 500 companies, research institutions, and Web development organizations that issues technical specifications for Web languages and protocols.

W3C specifications are called "recommendations" because the W3C is technically not a standards-issuing organization, but in practice this is usually an issue of semantics.

Recommendations are taken as defining a standard form of a Web language, and they are used by Web developers, software tools creators, browser programmers, and others as a blueprint for computer communication over the Web. Examples of W3C Recommendations include Hypertext Markup Language (HTML) and Extensible Markup Language (XML).

The CSS Specifications

The W3C Recommendations issued by the Cascading Style Sheet working group compose the official specification for the CSS language. The CSS working group consists of a number of experts in Web development, graphic design, and software programming, representing a number of companies, who all work together to establish a common styling language for the Web.

CSS Level 1

The Cascading Style Sheets Level 1 (sometimes called CSS1 for short) was officially issued as a W3C Recommendation in December 1996. The URL for this specification is
http://www.w3.org/TR/REC-CSS1.

If you try to read the W3C Recommendation for CSS1, you may end up confused. That's because W3C documents aren't written as a general introduction to a subject but rather as precise language definitions for software developers. Most W3C Recommendations are quite opaque to most normal people, although the CSS1 specification isn't too bad compared with some. Being able to refer to the official specification is quite useful, though.

Optional note for programmers - Classes and IDs

In addition to setting styles based on HTML elements, CSS allows for rules based on two optional attributes in HTML: class and id. Each of these can serve as the selector part of a CSS rule and can be set on any visible HTML tag.

The div and span elements really come into their own with class and id selectors. Through the use of class and id attributes, div or span tags can be made to have nearly any effect and presentation, which is often good but sometimes bad. Care must be taken to avoid using class or id selectors that you're not ignoring more appropriate markup, which has understood semantics. In other words, a div with a class of bldtxt has no specific meaning in the context of HTML, but a strong tag definitely does. Before using div or span, consider if another tag would make more sense.

The mention above of the word "semantics" is meant to be a segue to the subject of a later post, semantic interoperability.



Thursday, July 16, 2009

Customer self-service and social Web communities like Twitter and Facebook vs. spammers, scammers and hackers


Organizations from large corporations to small medical centers are bridging the gap between their cloud-computing customer service and support contact centers and social Web communities like Twitter and Facebook.

See, for example, https://www.salesforce.com/form/demo/csm_reg.jsp?d=70130000000EoNf

This means that there are new targets for spammers, scammers and hackers.

Redux

At the bottom of my May 30, 2009 post I added the following note on full-URL links vs. compressed links:

I've been asked why I didn't use link-shrinkers in earlier posts. Here's why:

First, I should say that there are some things I like about link compression: Some link-shrinkers let you personalize the new address with a unique phrase such as your name, or show you how many people click the link after you've posted it. Furthermore, link compression is just the beginning. More and more of these outfits allow users to see all sorts of details like where a link is showing up around the Web and where the people clicking on it are located.

However, this convenience may come at a cost. The tools add another layer to the process of navigating the Web, potentially leaving a trail of broken links if a service suddenly closes shop. They can also make it harder to tell what you're really clicking on, which may make these Lilliputian links attractive to spammers and scammers.

But popularity and convenience don't eliminate the potential risks of these link loppers. If so many services are springing up, chances are some will just as quickly disappear. And if a URL shortening service goes down, the links created with it could lead nowhere.

Another worry is that you're not likely to know exactly where a truncated link will take you. So you could be directed to unsavory or illegal content or something malicious like a computer worm. This means URL shortening services need to keep an eye on the kinds of sites their users are linking to.

Some problems

Purveyors of spam and malicious software are taking full advantage of URL-shortening services like bit.ly and TinyURL in a bid to trick unwary users into clicking on links to dodgy and dangerous Web sites. Fortunately, with the help of a couple of tools and some common sense, most Internet users can avoid these scams altogether.

According to alerts from anti-virus vendors McAfee, Symantec and Trend Micro, the latest to abuse these services is the Koobface worm, which targets users of social networking sites like Facebook (Koobface is an anagram of Facebook) and Myspace. It's now also spreading via microblogging service Twitter. Koobface arrives as a message that urges users to click on a link to a video, which invariably leads to a site that prompts the visitor to install a missing video plug-in. The fake plug-in turns the user's system into a bot that can be used for a variety of criminal purposes, from spamming to attacking other computers and spreading the worm.

Some solutions


TinyURL, which is among the longest-running URL shortening services, lets you automatically enable the preview of all shortened URLs. Just visit this page and click the "Enable Previews" link, and from then on TinyURLs will be converted into their longer form when you visit a Web page that features them. You must have cookies enabled in your browser for this setting to take, and you will need to set the cookie for each browser you use.

If you browse the Web with Firefox, there is an add-on called Long URL Please, which currently converts URLs shortened by 72 different services, including bit.ly, cli.gs, digg.com, is.gd, kl.am, ow.ly, tr.im, and tinyurl.com. Long_URL_Please also works in Internet Explorer and other browsers: Simply add their bookmark to your bookmarks, and then click on it when you're at a page that includes shortened URLs to display the long URL.

Firefox users who are familiar with the Greasemonkey add-on may prefer the Tiny URL Decoder script , which also works with a long list of URL shortening services.

Expandmyurl.com is another bookmarklet approach that works across browsers.