D-rule - Revision history

imported>Martins at 11:53, 30 January 2015

2015-01-30T11:53:22Z

← Older revision		Revision as of 11:53, 30 January 2015
Line 87:		Line 87:

	== #PREFERRED AND #FINAL ==		== #PREFERRED AND #FINAL ==
	The features #PREFERRED and #FINAL are used to change the default order of replacements during [[tokenization]] ~~when the best match is blocked:~~		The features #PREFERRED and #FINAL are used to change the default order of replacements during [[tokenization]] in blocking rules.
	*<nowiki>#PREFERRED</nowiki> means "if this possibility is blocked, <b>try first other candidate tokens for the other strings</b> before trying other candidate tokens for this string"		*<nowiki>#PREFERRED</nowiki> means "if this possibility is blocked, <b>try first other candidate tokens for the other strings</b> before trying other candidate tokens for this string"
	*<nowiki>#FINAL</nowiki> means "if this possibility is blocked, <b>do not try any other candidate token for this string</b>, i.e., try only other candidate tokens for the other strings"		*<nowiki>#FINAL</nowiki> means "if this possibility is blocked, <b>do not try any other candidate token for this string</b>, i.e., try only other candidate tokens for the other strings"
Line 113:		Line 113:
	Compare the behaviour of the system for the grammars below:		Compare the behaviour of the system for the grammars below:
	*D-GRAMMAR #1		*D-GRAMMAR #1
	#<nowiki>(A1)(B1)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B1)		#<nowiki>(A1)(B1,#PREFERRED)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B1, but indicates that A1 must be changed prior to B1)
	~~#<nowiki>("a")(~~B1~~,#PREFERRED)=0;</nowiki> (alter the order of best matches~~)
	*NEW ORDER		*NEW ORDER
	#<nowiki>[A2][B1][C1]</nowiki> (previously 5)		#<nowiki>[A2][B1][C1]</nowiki> (previously 5)
Line 124:		Line 123:
	Note, in the above, that the options 1 and 2 are not tested because they are blocked, and the order of attempts is changed. However, if, for any reason (other blocking rules), these first alternatives are also blocked, the system still tries to find, within the same segmentation, other candidates. If all of them are blocked, and only then, the system will re-segment the string, according to the process defined at [[tokenization]].		Note, in the above, that the options 1 and 2 are not tested because they are blocked, and the order of attempts is changed. However, if, for any reason (other blocking rules), these first alternatives are also blocked, the system still tries to find, within the same segmentation, other candidates. If all of them are blocked, and only then, the system will re-segment the string, according to the process defined at [[tokenization]].
	*D-GRAMMAR #2		*D-GRAMMAR #2
	#<nowiki>(A1)(B1)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B1)		#<nowiki>(A1)(B1,#FINAL)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B1 and prevents the system from trying other candidates for B1)
	~~#<nowiki>("a")(B1,#FINAL)=0;</nowiki> (blocks~~ other candidates for ~~the second node~~)
	*NEW ORDER		*NEW ORDER
	#<nowiki>[A2][B1][C1]</nowiki> (previously 5)		#<nowiki>[A2][B1][C1]</nowiki> (previously 5)
	#<nowiki>[A2][B1][C2]</nowiki> (previously 6)		#<nowiki>[A2][B1][C2]</nowiki> (previously 6)
	Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />		Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />
	At last, note that ~~neither~~ #PREFERRED ~~nor~~ #FINAL ~~prevails over~~ the ~~default tokenization order IF NO BLOCKING RULE IS MATCHED, i~~.~~e., if~~ the grammar was simply:		At last, note that #PREFERRED and #FINAL do not prevail other the priority of application of rules. If the grammar was simply:
	#<nowiki>(A1~~)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B2)~~		#<nowiki>(A1)(B2,#PREFERRED)=0;</nowiki>
	~~#<nowiki>("a"~~)(B2,#PREFERRED)=0;</nowiki> ~~(alter the order of best matches)~~
	or		or
	#<nowiki>(A1~~)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B2~~		#<nowiki>(A1)(B2,#FINAL)=0;</nowiki>
	~~#<nowiki>("a"~~)(B2,#FINAL)=0;</nowiki> ~~(alter~~ the ~~order of best matches)~~		the output would be still
	#<nowiki>[A1][B1][C1]</nowiki>		#<nowiki>[A1][B1][C1]</nowiki>
	~~i.e., #FINAL and #PREFERRED only play a role when~~ the ~~default~~ best match is blocked.		because this is the best match and it is not blocked by any rule.

	For a practical example in English, see below<ref>		For a practical example in English, see below<ref>

imported>Martins at 11:24, 30 January 2015

2015-01-30T11:24:17Z

← Older revision		Revision as of 11:24, 30 January 2015
Line 131:		Line 131:
	Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />		Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />
	At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:		At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:
	#<nowiki>(A1)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B1)		#<nowiki>(A1)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B2)
	#<nowiki>("a")(B2,#PREFERRED)=0;</nowiki> (alter the order of best matches)		#<nowiki>("a")(B2,#PREFERRED)=0;</nowiki> (alter the order of best matches)
	or		or
	#<nowiki>(A1)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and ~~B1)~~		#<nowiki>(A1)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B2
	#<nowiki>("a")(B2,#~~PREFERRED~~)=0;</nowiki> (alter the order of best matches)		#<nowiki>("a")(B2,#FINAL)=0;</nowiki> (alter the order of best matches)
	#<nowiki>[A1][B1][C1]</nowiki>		#<nowiki>[A1][B1][C1]</nowiki>
	i.e., #FINAL and #PREFERRED only play a role when the default best match is blocked.		i.e., #FINAL and #PREFERRED only play a role when the default best match is blocked.

imported>Martins at 11:03, 30 January 2015

2015-01-30T11:03:01Z

← Older revision		Revision as of 11:03, 30 January 2015
Line 156:		Line 156:
	because D-rules apply from left to right, and the system will try to replace first the rightmost nodes, if possible.<br />		because D-rules apply from left to right, and the system will try to replace first the rightmost nodes, if possible.<br />
	In order to prevent the system from replacing the rightmost nodes, we have to assign #PREFERRED to the nodes to be preserved:		In order to prevent the system from replacing the rightmost nodes, we have to assign #PREFERRED to the nodes to be preserved:
	*(R)(N,#PREFERRED)=0; (there cannot be a pronoun before a noun)		*(R)(N,#PREFERRED)=0; (there cannot be a pronoun before a noun, but try to replace the pronoun before trying to replace the noun)
	In this case, the machine will try to replace first the node without #PREFERRED and will get, then:<br />		In this case, the machine will try to replace first the node without #PREFERRED and will get, then:<br />
	*(D)(N) (i.e., [this] = determiner and [book] = noun)<br />		*(D)(N) (i.e., [this] = determiner and [book] = noun)<br />

imported>Martins at 11:01, 30 January 2015

2015-01-30T11:01:43Z

← Older revision		Revision as of 11:01, 30 January 2015
Line 131:		Line 131:
	Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />		Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />
	At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:		At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:
	#<nowiki>("a")(B1,#PREFERRED)=0;</nowiki> (alter the order of best matches)		#<nowiki>(A1)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B1)
			#<nowiki>("a")(B2,#PREFERRED)=0;</nowiki> (alter the order of best matches)
	or		or
	#<nowiki>(~~"a"~~)(~~B1,#FINAL~~)=0;</nowiki> (blocks ~~other candidates for~~ the ~~second node~~)		#<nowiki>(A1)(B2)=0;</nowiki> (blocks the sequence of tokens with the features A1 and B1)
	~~i.e., without~~		#<nowiki>("a")(B2,#PREFERRED)=0;</nowiki> (alter the order of best matches)
	#<nowiki>(A1)(B1)=0;</nowiki>
	the ~~final output would be:~~
	#<nowiki>[A1][B1][C1]</nowiki>		#<nowiki>[A1][B1][C1]</nowiki>
	i.e., #FINAL and #PREFERRED only play a role when the default best match is blocked.		i.e., #FINAL and #PREFERRED only play a role when the default best match is blocked.

imported>Martins at 10:59, 30 January 2015

2015-01-30T10:59:45Z

← Older revision		Revision as of 10:59, 30 January 2015
Line 138:		Line 138:
	the final output would be:		the final output would be:
	#<nowiki>[A1][B1][C1]</nowiki>		#<nowiki>[A1][B1][C1]</nowiki>
			i.e., #FINAL and #PREFERRED only play a role when the default best match is blocked.

	For a practical example in English, see below<ref>		For a practical example in English, see below<ref>
	Input string:		Input string:

imported>Martins at 10:58, 30 January 2015

2015-01-30T10:58:40Z

← Older revision		Revision as of 10:58, 30 January 2015
Line 130:		Line 130:
	#<nowiki>[A2][B1][C2]</nowiki> (previously 6)		#<nowiki>[A2][B1][C2]</nowiki> (previously 6)
	Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />		Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />
	At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO ~~OTHER~~ BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:		At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:
	#<nowiki>("a")(B1,#PREFERRED)=0;</nowiki> (alter the order of best matches)		#<nowiki>("a")(B1,#PREFERRED)=0;</nowiki> (alter the order of best matches)
	or		or

imported>Martins at 10:56, 30 January 2015

2015-01-30T10:56:35Z

← Older revision		Revision as of 10:56, 30 January 2015
Line 99:		Line 99:
	**<nowiki>[C]{}"c1"(C1)<,,>;</nowiki>		**<nowiki>[C]{}"c1"(C1)<,,>;</nowiki>
	**<nowiki>[C]{}"c2"(C2)<,,>;</nowiki>		**<nowiki>[C]{}"c2"(C2)<,,>;</nowiki>
	According to the default order (left to right), the tokenization will try the following alternatives (in this specific order)~~, if no disambiguation rule is provided~~:		According to the default order (left to right), the tokenization will try the following alternatives (in this specific order):
	#<nowiki>[A1][B1][C1]</nowiki>		#<nowiki>[A1][B1][C1]</nowiki>
	#<nowiki>[A1][B1][C2]</nowiki>		#<nowiki>[A1][B1][C2]</nowiki>

imported>Martins at 10:56, 30 January 2015

2015-01-30T10:56:14Z

← Older revision		Revision as of 10:56, 30 January 2015
Line 99:		Line 99:
	**<nowiki>[C]{}"c1"(C1)<,,>;</nowiki>		**<nowiki>[C]{}"c1"(C1)<,,>;</nowiki>
	**<nowiki>[C]{}"c2"(C2)<,,>;</nowiki>		**<nowiki>[C]{}"c2"(C2)<,,>;</nowiki>
	According to the default order (left to right), the tokenization will try the following alternatives (in ~~that~~ specific order), if no disambiguation rule is provided:		According to the default order (left to right), the tokenization will try the following alternatives (in this specific order), if no disambiguation rule is provided:
	#<nowiki>[A1][B1][C1]</nowiki>		#<nowiki>[A1][B1][C1]</nowiki>
	#<nowiki>[A1][B1][C2]</nowiki>		#<nowiki>[A1][B1][C2]</nowiki>

imported>Martins at 10:38, 30 January 2015

2015-01-30T10:38:15Z

← Older revision		Revision as of 10:38, 30 January 2015
Line 129:		Line 129:
	#<nowiki>[A2][B1][C1]</nowiki> (previously 5)		#<nowiki>[A2][B1][C1]</nowiki> (previously 5)
	#<nowiki>[A2][B1][C2]</nowiki> (previously 6)		#<nowiki>[A2][B1][C2]</nowiki> (previously 6)
	Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.		Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.<br />
	At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO OTHER BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:		At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO OTHER BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:
	#<nowiki>("a")(B1,#PREFERRED)=0;</nowiki> (alter the order of best matches)		#<nowiki>("a")(B1,#PREFERRED)=0;</nowiki> (alter the order of best matches)

imported>Martins at 10:37, 30 January 2015

2015-01-30T10:37:33Z

← Older revision		Revision as of 10:37, 30 January 2015
Line 131:		Line 131:
	Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.		Note, in the above, that the options 1 and 2 are not tested because they are blocked. Note, also, that other candidates for the second string (i.e., where the second string is B2) are not tested, because B1 is #FINAL. So, if these two matches fail, the system starts re-segmenting the sentence.
	At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO OTHER BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:		At last, note that neither #PREFERRED nor #FINAL prevails over the default tokenization order IF NO OTHER BLOCKING RULE IS MATCHED, i.e., if the grammar was simply:
	#<nowiki>("a")(B1,#PREFERRED)=0;<nowiki> (alter the order of best matches)		#<nowiki>("a")(B1,#PREFERRED)=0;</nowiki> (alter the order of best matches)
	or		or
	#<nowiki>("a")(B1,#FINAL)=0;</nowiki> (blocks other candidates for the second node)		#<nowiki>("a")(B1,#FINAL)=0;</nowiki> (blocks other candidates for the second node)