imported>Martins at 18:45, 4 August 2014

2014-08-04T18:45:29Z

New page

CLEA2000 certifies that you have reached a standard of knowledge concerning the development of natural language grammars for [[NLization]] and may officially participate in the UNL Research. The certificate consists of a set of 100 graphs that must be NLized with EUGENE.

== Goal ==
In CLEA2000, you are expected to provide the lingware (dictionary and grammars) necessary to '''NLize''' the corpus [[UCA1]] into your native language.

== Instructions ==
#Training corpus
##Prepare the training corpus
##*If your native language is not English: translate (manually) the 100 sentences of the corpus [[UCA1]] from English into your native language. Be as close as possible to the original, and provide one single translation for each sentence. This will be your expected output, and your goal will be to get (automatically, through [[EUGENE]]) these NL sentences out of the UNL graphs. See an example of input file [http://www.unlweb.net.br/resources/UCA1/uca1_unl.txt here]. Note that you have to translate only the sentences between {org}{/org}
##*If your native language is English: use the corpus [[UCA2]] as your input document file.
##Save the translated text (without the English original) in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>EUGENE>UNL FILES.
#Dictionary<ref>Instead of creating a whole new dictionary from scratch, you may try localizing the English dictionary available at [http://www.unlweb.net.br/resources/UCA1/unl_eng_dic.txt unl_eng_dic.txt]. Note that "localization" is not the same as "translation". You may need other features (in English, for instance, nouns do not have gender or case) or other entries. In any case, the resulting dictionary should reflect your translated version of the corpus (i.e., all entries appearing in your translated version of the corpus should appear in the dictionary). For further information on localization, see [[Localization]]. For information on the dictionary structure, see [[Dictonary Specs]]. For an explanation of the structure of the English dictionary, see [[English_dictionary|English Dictionary]]. In case you need additional features, use only the tags available at the [[tagset]].</ref>
##Create a dictionary for your training corpus. Your dictionary must contain all and only the word forms appearing in your training corpus, and must comply with the [[Dictionary Specs]].
##Save the NL-UNL dictionary in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>EUGENE>DICTIONARIES.
#Grammars<ref>In order to prepare the grammar, study the [[Grammar Specs]]. Next, take a look at the structure of the [[English_grammar|English Grammar]] for a detailed example. In many cases, it is simpler just to localize the English grammars to your own locale rather than creating a whole grammar from the scratch. See the instructions at [[Localization]].</ref>
##Create the UNL-NL (generation) grammars necessary to NLize the UNL graphs. These grammars are the most difficult (and the actual goal) of the whole analysis task. There can be two different types of grammar:
##*[[T-grammar]], which is used to transform the graph structure (the UNL graph) into a list structure (the NL sentence);
##*[[D-grammar]], which is used to improve the performance of the [[tokenization]] and of the [[transformation]].<ref>D-rules used in tokenization are used either to prevent wrong lexical matches or to provoke right lexical matches. D-rules used in transformation are used to either prevent or provoke the application of certain rules.</ref>
##Save the UNL-NL grammars in a plain text (.txt) file with UTF-8 encoding and upload them to the corresponding tabs in UNLWEB>UNLDEV>EUGENE.
##Test them and do the necessary changes until you get good results.
#Evaluation
##Export the actual output of EUGENE (range=1-100, trace-level=MINIMAL).
##Check the F-measure of the actual output at UNLWEB>UNLARIUM>TOOLS>F-MEASURE.<ref>The F-measure compares the actual output (exported from EUGENE) with the expected output (available at [http://www.unlweb.net.br/resources/UCA1/uca1_unl.txt uca1_unl.txt])</ref>
##If the F-measure > 0.9, upload the corpus, dictionary and grammars to VALERIE (UNLWEB>VALERIE>CLEA2000) in order to have your exercise evaluated.

== Samples and Examples ==
The following resources have been used to deal with UCA1 in English and may be used as a sample of what is expected to be provided
{|border=1 cellpadding=5 align=center
|+NLization
!Language
!Corpus
!Dictionary
!T-Grammar
!D-Grammar
!Output
|-
|align=center|English
|[http://www.unlweb.net.br/resources/UCA1/uca1_unl.txt uca1_unl.txt]
|[http://www.unlweb.net.br/resources/UCA1/unl_eng_dic.txt unl_eng_dic.txt]
|[http://www.unlweb.net.br/resources/UCA1/unl_eng_tgrammar.txt unl_eng_tgrammar.txt]
|[http://www.unlweb.net.br/resources/UCA1/unl_eng_dgrammar.txt unl_eng_dgrammar.txt]
|[http://www.unlweb.net.br/resources/UCA1/unl_eng_trace.txt unl_eng_trace.txt]
|}
 

== Recommended Readings ==
Before starting the activity, and in order to fully understand what is expected to be done, it is important for you to be acquainted with the following documentation:
*[[Tagset]], because you are expected to use only the tags included in the tagset
*[[UNL Dictionary Specs]], which is essential to understand the dictionary structure
*[[UNL Grammar Specs]], which is essential to understand the grammar structure
*[[X-bar]]
*[[Default Dictionary]]
*[[English dictionary]]
*[[Default grammar]]
*[[English grammar]]
It is also interesting to make a test drive with [[EUGENE]].

== Notes ==
<references/>

CLEA2000 - Revision history

imported>Martins at 18:45, 4 August 2014