Automatic text processing - Linguistic techniques

Documents available in electronic form are a major source of information, and are driving the development of applications designed to facilitate their management and use. This textual data is very diverse in nature:

• documentation produced by the company, its partners and customers (technical reports, maintenance documentation, contracts, meeting minutes, e-mail messages, etc.);

• information of a technological and economic nature that companies need to collect and exploit from a wide and diversified documentary environment (patents, research reports, grey literature, commercial and technical news available on the web, etc.).

It is through these documents that the bulk of information flows, and it is therefore crucial for organizations to have techniques for accessing the business knowledge contained in this data. In fact, most strategic information is textual in nature. Understanding and analyzing it is essential for :

• scientific and technological monitoring, knowledge management and transfer;

• support decision-making, risk identification, etc.

And yet, this data is both voluminous and unstructured. It is highly heterogeneous in nature. It is rarely written to explicit standards, and may be written under time pressure (reports, notes, minutes, letters). These characteristics make the material very difficult to process: relevant information has to be extracted from the textual flow; this extraction is complex due to the ambiguity and variability that characterize language expression. The exploitation of these "off-the-shelf" texts has therefore become a major technological challenge. New technical solutions, often described as "semantic" and "intelligent", are being proposed to companies to :

• mastering the profusion of electronic documents – procedures for classifying, selecting, synthesizing and structuring documents;

• extract and organize the information they contain.

These solutions are based on automatic language processing (ALP) techniques. The aim of this dossier is to provide an overview of the automated language processing techniques used and, by facilitating understanding of these techniques, to enable a reasoned choice to be made among the solutions proposed in the field of information processing.