diff options
Diffstat (limited to 'lib/win32/pcre/doc/html/pcrepartial.html')
| -rw-r--r-- | lib/win32/pcre/doc/html/pcrepartial.html | 400 |
1 files changed, 400 insertions, 0 deletions
diff --git a/lib/win32/pcre/doc/html/pcrepartial.html b/lib/win32/pcre/doc/html/pcrepartial.html new file mode 100644 index 0000000000..040ac88f30 --- /dev/null +++ b/lib/win32/pcre/doc/html/pcrepartial.html @@ -0,0 +1,400 @@ +<html> +<head> +<title>pcrepartial specification</title> +</head> +<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> +<h1>pcrepartial man page</h1> +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> +<p> +This page is part of the PCRE HTML documentation. It was generated automatically +from the original man page. If there is any nonsense in it, please consult the +man page, in case the conversion went wrong. +<br> +<ul> +<li><a name="TOC1" href="#SEC1">PARTIAL MATCHING IN PCRE</a> +<li><a name="TOC2" href="#SEC2">PARTIAL MATCHING USING pcre_exec()</a> +<li><a name="TOC3" href="#SEC3">PARTIAL MATCHING USING pcre_dfa_exec()</a> +<li><a name="TOC4" href="#SEC4">PARTIAL MATCHING AND WORD BOUNDARIES</a> +<li><a name="TOC5" href="#SEC5">FORMERLY RESTRICTED PATTERNS</a> +<li><a name="TOC6" href="#SEC6">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a> +<li><a name="TOC7" href="#SEC7">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec()</a> +<li><a name="TOC8" href="#SEC8">MULTI-SEGMENT MATCHING WITH pcre_exec()</a> +<li><a name="TOC9" href="#SEC9">ISSUES WITH MULTI-SEGMENT MATCHING</a> +<li><a name="TOC10" href="#SEC10">AUTHOR</a> +<li><a name="TOC11" href="#SEC11">REVISION</a> +</ul> +<br><a name="SEC1" href="#TOC1">PARTIAL MATCHING IN PCRE</a><br> +<P> +In normal use of PCRE, if the subject string that is passed to +<b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> matches as far as it goes, but is +too short to match the entire pattern, PCRE_ERROR_NOMATCH is returned. There +are circumstances where it might be helpful to distinguish this case from other +cases in which there is no match. +</P> +<P> +Consider, for example, an application where a human is required to type in data +for a field with specific formatting requirements. An example might be a date +in the form <i>ddmmmyy</i>, defined by this pattern: +<pre> + ^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$ +</pre> +If the application sees the user's keystrokes one by one, and can check that +what has been typed so far is potentially valid, it is able to raise an error +as soon as a mistake is made, by beeping and not reflecting the character that +has been typed, for example. This immediate feedback is likely to be a better +user interface than a check that is delayed until the entire string has been +entered. Partial matching can also sometimes be useful when the subject string +is very long and is not all available at once. +</P> +<P> +PCRE supports partial matching by means of the PCRE_PARTIAL_SOFT and +PCRE_PARTIAL_HARD options, which can be set when calling <b>pcre_exec()</b> or +<b>pcre_dfa_exec()</b>. For backwards compatibility, PCRE_PARTIAL is a synonym +for PCRE_PARTIAL_SOFT. The essential difference between the two options is +whether or not a partial match is preferred to an alternative complete match, +though the details differ between the two matching functions. If both options +are set, PCRE_PARTIAL_HARD takes precedence. +</P> +<P> +Setting a partial matching option disables two of PCRE's optimizations. PCRE +remembers the last literal byte in a pattern, and abandons matching immediately +if such a byte is not present in the subject string. This optimization cannot +be used for a subject string that might match only partially. If the pattern +was studied, PCRE knows the minimum length of a matching string, and does not +bother to run the matching function on shorter strings. This optimization is +also disabled for partial matching. +</P> +<br><a name="SEC2" href="#TOC1">PARTIAL MATCHING USING pcre_exec()</a><br> +<P> +A partial match occurs during a call to <b>pcre_exec()</b> whenever the end of +the subject string is reached successfully, but matching cannot continue +because more characters are needed. However, at least one character must have +been matched. (In other words, a partial match can never be an empty string.) +</P> +<P> +If PCRE_PARTIAL_SOFT is set, the partial match is remembered, but matching +continues as normal, and other alternatives in the pattern are tried. If no +complete match can be found, <b>pcre_exec()</b> returns PCRE_ERROR_PARTIAL +instead of PCRE_ERROR_NOMATCH. If there are at least two slots in the offsets +vector, the first of them is set to the offset of the earliest character that +was inspected when the partial match was found. For convenience, the second +offset points to the end of the string so that a substring can easily be +identified. +</P> +<P> +For the majority of patterns, the first offset identifies the start of the +partially matched string. However, for patterns that contain lookbehind +assertions, or \K, or begin with \b or \B, earlier characters have been +inspected while carrying out the match. For example: +<pre> + /(?<=abc)123/ +</pre> +This pattern matches "123", but only if it is preceded by "abc". If the subject +string is "xyzabc12", the offsets after a partial match are for the substring +"abc12", because all these characters are needed if another match is tried +with extra characters added. +</P> +<P> +If there is more than one partial match, the first one that was found provides +the data that is returned. Consider this pattern: +<pre> + /123\w+X|dogY/ +</pre> +If this is matched against the subject string "abc123dog", both +alternatives fail to match, but the end of the subject is reached during +matching, so PCRE_ERROR_PARTIAL is returned instead of PCRE_ERROR_NOMATCH. The +offsets are set to 3 and 9, identifying "123dog" as the first partial match +that was found. (In this example, there are two partial matches, because "dog" +on its own partially matches the second alternative.) +</P> +<P> +If PCRE_PARTIAL_HARD is set for <b>pcre_exec()</b>, it returns +PCRE_ERROR_PARTIAL as soon as a partial match is found, without continuing to +search for possible complete matches. The difference between the two options +can be illustrated by a pattern such as: +<pre> + /dog(sbody)?/ +</pre> +This matches either "dog" or "dogsbody", greedily (that is, it prefers the +longer string if possible). If it is matched against the string "dog" with +PCRE_PARTIAL_SOFT, it yields a complete match for "dog". However, if +PCRE_PARTIAL_HARD is set, the result is PCRE_ERROR_PARTIAL. On the other hand, +if the pattern is made ungreedy the result is different: +<pre> + /dog(sbody)??/ +</pre> +In this case the result is always a complete match because <b>pcre_exec()</b> +finds that first, and it never continues after finding a match. It might be +easier to follow this explanation by thinking of the two patterns like this: +<pre> + /dog(sbody)?/ is the same as /dogsbody|dog/ + /dog(sbody)??/ is the same as /dog|dogsbody/ +</pre> +The second pattern will never match "dogsbody" when <b>pcre_exec()</b> is +used, because it will always find the shorter match first. +</P> +<br><a name="SEC3" href="#TOC1">PARTIAL MATCHING USING pcre_dfa_exec()</a><br> +<P> +The <b>pcre_dfa_exec()</b> function moves along the subject string character by +character, without backtracking, searching for all possible matches +simultaneously. If the end of the subject is reached before the end of the +pattern, there is the possibility of a partial match, again provided that at +least one character has matched. +</P> +<P> +When PCRE_PARTIAL_SOFT is set, PCRE_ERROR_PARTIAL is returned only if there +have been no complete matches. Otherwise, the complete matches are returned. +However, if PCRE_PARTIAL_HARD is set, a partial match takes precedence over any +complete matches. The portion of the string that was inspected when the longest +partial match was found is set as the first matching string, provided there are +at least two slots in the offsets vector. +</P> +<P> +Because <b>pcre_dfa_exec()</b> always searches for all possible matches, and +there is no difference between greedy and ungreedy repetition, its behaviour is +different from <b>pcre_exec</b> when PCRE_PARTIAL_HARD is set. Consider the +string "dog" matched against the ungreedy pattern shown above: +<pre> + /dog(sbody)??/ +</pre> +Whereas <b>pcre_exec()</b> stops as soon as it finds the complete match for +"dog", <b>pcre_dfa_exec()</b> also finds the partial match for "dogsbody", and +so returns that when PCRE_PARTIAL_HARD is set. +</P> +<br><a name="SEC4" href="#TOC1">PARTIAL MATCHING AND WORD BOUNDARIES</a><br> +<P> +If a pattern ends with one of sequences \b or \B, which test for word +boundaries, partial matching with PCRE_PARTIAL_SOFT can give counter-intuitive +results. Consider this pattern: +<pre> + /\bcat\b/ +</pre> +This matches "cat", provided there is a word boundary at either end. If the +subject string is "the cat", the comparison of the final "t" with a following +character cannot take place, so a partial match is found. However, +<b>pcre_exec()</b> carries on with normal matching, which matches \b at the end +of the subject when the last character is a letter, thus finding a complete +match. The result, therefore, is <i>not</i> PCRE_ERROR_PARTIAL. The same thing +happens with <b>pcre_dfa_exec()</b>, because it also finds the complete match. +</P> +<P> +Using PCRE_PARTIAL_HARD in this case does yield PCRE_ERROR_PARTIAL, because +then the partial match takes precedence. +</P> +<br><a name="SEC5" href="#TOC1">FORMERLY RESTRICTED PATTERNS</a><br> +<P> +For releases of PCRE prior to 8.00, because of the way certain internal +optimizations were implemented in the <b>pcre_exec()</b> function, the +PCRE_PARTIAL option (predecessor of PCRE_PARTIAL_SOFT) could not be used with +all patterns. From release 8.00 onwards, the restrictions no longer apply, and +partial matching with <b>pcre_exec()</b> can be requested for any pattern. +</P> +<P> +Items that were formerly restricted were repeated single characters and +repeated metasequences. If PCRE_PARTIAL was set for a pattern that did not +conform to the restrictions, <b>pcre_exec()</b> returned the error code +PCRE_ERROR_BADPARTIAL (-13). This error code is no longer in use. The +PCRE_INFO_OKPARTIAL call to <b>pcre_fullinfo()</b> to find out if a compiled +pattern can be used for partial matching now always returns 1. +</P> +<br><a name="SEC6" href="#TOC1">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a><br> +<P> +If the escape sequence \P is present in a <b>pcretest</b> data line, the +PCRE_PARTIAL_SOFT option is used for the match. Here is a run of <b>pcretest</b> +that uses the date example quoted above: +<pre> + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 25jun04\P + 0: 25jun04 + 1: jun + data> 25dec3\P + Partial match: 23dec3 + data> 3ju\P + Partial match: 3ju + data> 3juj\P + No match + data> j\P + No match +</pre> +The first data string is matched completely, so <b>pcretest</b> shows the +matched substrings. The remaining four strings do not match the complete +pattern, but the first two are partial matches. Similar output is obtained +when <b>pcre_dfa_exec()</b> is used. +</P> +<P> +If the escape sequence \P is present more than once in a <b>pcretest</b> data +line, the PCRE_PARTIAL_HARD option is set for the match. +</P> +<br><a name="SEC7" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec()</a><br> +<P> +When a partial match has been found using <b>pcre_dfa_exec()</b>, it is possible +to continue the match by providing additional subject data and calling +<b>pcre_dfa_exec()</b> again with the same compiled regular expression, this +time setting the PCRE_DFA_RESTART option. You must pass the same working +space as before, because this is where details of the previous partial match +are stored. Here is an example using <b>pcretest</b>, using the \R escape +sequence to set the PCRE_DFA_RESTART option (\D specifies the use of +<b>pcre_dfa_exec()</b>): +<pre> + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 23ja\P\D + Partial match: 23ja + data> n05\R\D + 0: n05 +</pre> +The first call has "23ja" as the subject, and requests partial matching; the +second call has "n05" as the subject for the continued (restarted) match. +Notice that when the match is complete, only the last part is shown; PCRE does +not retain the previously partially-matched string. It is up to the calling +program to do that if it needs to. +</P> +<P> +You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with +PCRE_DFA_RESTART to continue partial matching over multiple segments. This +facility can be used to pass very long subject strings to +<b>pcre_dfa_exec()</b>. +</P> +<br><a name="SEC8" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_exec()</a><br> +<P> +From release 8.00, <b>pcre_exec()</b> can also be used to do multi-segment +matching. Unlike <b>pcre_dfa_exec()</b>, it is not possible to restart the +previous match with a new segment of data. Instead, new data must be added to +the previous subject string, and the entire match re-run, starting from the +point where the partial match occurred. Earlier data can be discarded. +Consider an unanchored pattern that matches dates: +<pre> + re> /\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d/ + data> The date is 23ja\P + Partial match: 23ja +</pre> +At this stage, an application could discard the text preceding "23ja", add on +text from the next segment, and call <b>pcre_exec()</b> again. Unlike +<b>pcre_dfa_exec()</b>, the entire matching string must always be available, and +the complete matching process occurs for each call, so more memory and more +processing time is needed. +</P> +<P> +<b>Note:</b> If the pattern contains lookbehind assertions, or \K, or starts +with \b or \B, the string that is returned for a partial match will include +characters that precede the partially matched string itself, because these must +be retained when adding on more characters for a subsequent matching attempt. +</P> +<br><a name="SEC9" href="#TOC1">ISSUES WITH MULTI-SEGMENT MATCHING</a><br> +<P> +Certain types of pattern may give problems with multi-segment matching, +whichever matching function is used. +</P> +<P> +1. If the pattern contains tests for the beginning or end of a line, you need +to pass the PCRE_NOTBOL or PCRE_NOTEOL options, as appropriate, when the +subject string for any call does not contain the beginning or end of a line. +</P> +<P> +2. Lookbehind assertions at the start of a pattern are catered for in the +offsets that are returned for a partial match. However, in theory, a lookbehind +assertion later in the pattern could require even earlier characters to be +inspected, and it might not have been reached when a partial match occurs. This +is probably an extremely unlikely case; you could guard against it to a certain +extent by always including extra characters at the start. +</P> +<P> +3. Matching a subject string that is split into multiple segments may not +always produce exactly the same result as matching over one single long string, +especially when PCRE_PARTIAL_SOFT is used. The section "Partial Matching and +Word Boundaries" above describes an issue that arises if the pattern ends with +\b or \B. Another kind of difference may occur when there are multiple +matching possibilities, because a partial match result is given only when there +are no completed matches. This means that as soon as the shortest match has +been found, continuation to a new subject segment is no longer possible. +Consider again this <b>pcretest</b> example: +<pre> + re> /dog(sbody)?/ + data> dogsb\P + 0: dog + data> do\P\D + Partial match: do + data> gsb\R\P\D + 0: g + data> dogsbody\D + 0: dogsbody + 1: dog +</pre> +The first data line passes the string "dogsb" to <b>pcre_exec()</b>, setting the +PCRE_PARTIAL_SOFT option. Although the string is a partial match for +"dogsbody", the result is not PCRE_ERROR_PARTIAL, because the shorter string +"dog" is a complete match. Similarly, when the subject is presented to +<b>pcre_dfa_exec()</b> in several parts ("do" and "gsb" being the first two) the +match stops when "dog" has been found, and it is not possible to continue. On +the other hand, if "dogsbody" is presented as a single string, +<b>pcre_dfa_exec()</b> finds both matches. +</P> +<P> +Because of these problems, it is probably best to use PCRE_PARTIAL_HARD when +matching multi-segment data. The example above then behaves differently: +<pre> + re> /dog(sbody)?/ + data> dogsb\P\P + Partial match: dogsb + data> do\P\D + Partial match: do + data> gsb\R\P\P\D + Partial match: gsb + +</PRE> +</P> +<P> +4. Patterns that contain alternatives at the top level which do not all +start with the same pattern item may not work as expected when +PCRE_DFA_RESTART is used with <b>pcre_dfa_exec()</b>. For example, consider this +pattern: +<pre> + 1234|3789 +</pre> +If the first part of the subject is "ABC123", a partial match of the first +alternative is found at offset 3. There is no partial match for the second +alternative, because such a match does not start at the same point in the +subject string. Attempting to continue with the string "7890" does not yield a +match because only those alternatives that match at one point in the subject +are remembered. The problem arises because the start of the second alternative +matches within the first alternative. There is no problem with anchored +patterns or patterns such as: +<pre> + 1234|ABCD +</pre> +where no string can be a partial match for both alternatives. This is not a +problem if <b>pcre_exec()</b> is used, because the entire match has to be rerun +each time: +<pre> + re> /1234|3789/ + data> ABC123\P + Partial match: 123 + data> 1237890 + 0: 3789 +</pre> +Of course, instead of using PCRE_DFA_PARTIAL, the same technique of re-running +the entire match can also be used with <b>pcre_dfa_exec()</b>. Another +possibility is to work with two buffers. If a partial match at offset <i>n</i> +in the first buffer is followed by "no match" when PCRE_DFA_RESTART is used on +the second buffer, you can then try a new match starting at offset <i>n+1</i> in +the first buffer. +</P> +<br><a name="SEC10" href="#TOC1">AUTHOR</a><br> +<P> +Philip Hazel +<br> +University Computing Service +<br> +Cambridge CB2 3QH, England. +<br> +</P> +<br><a name="SEC11" href="#TOC1">REVISION</a><br> +<P> +Last updated: 19 October 2009 +<br> +Copyright © 1997-2009 University of Cambridge. +<br> +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> |