757 lines · plain
1.\" $OpenBSD: re_format.7,v 1.14 2007/05/31 19:19:30 jmc Exp $2.\"3.\" Copyright (c) 1997, Phillip F Knaack. All rights reserved.4.\"5.\" Copyright (c) 1992, 1993, 1994 Henry Spencer.6.\" Copyright (c) 1992, 1993, 19947.\" The Regents of the University of California. All rights reserved.8.\"9.\" This code is derived from software contributed to Berkeley by10.\" Henry Spencer.11.\"12.\" Redistribution and use in source and binary forms, with or without13.\" modification, are permitted provided that the following conditions14.\" are met:15.\" 1. Redistributions of source code must retain the above copyright16.\" notice, this list of conditions and the following disclaimer.17.\" 2. Redistributions in binary form must reproduce the above copyright18.\" notice, this list of conditions and the following disclaimer in the19.\" documentation and/or other materials provided with the distribution.20.\" 3. Neither the name of the University nor the names of its contributors21.\" may be used to endorse or promote products derived from this software22.\" without specific prior written permission.23.\"24.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND25.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE26.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE27.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE28.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL29.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS30.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)31.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT32.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY33.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF34.\" SUCH DAMAGE.35.\"36.\" @(#)re_format.7 8.3 (Berkeley) 3/20/9437.\"38.Dd $Mdocdate: May 31 2007 $39.Dt RE_FORMAT 740.Os41.Sh NAME42.Nm re_format43.Nd POSIX regular expressions44.Sh DESCRIPTION45Regular expressions (REs),46as defined in47.St -p1003.1-2004 ,48come in two forms:49basic regular expressions50(BREs)51and extended regular expressions52(EREs).53Both forms of regular expressions are supported54by the interfaces described in55.Xr regex 3 .56Applications dealing with regular expressions57may use one or the other form58(or indeed both).59For example,60.Xr ed 161uses BREs,62whilst63.Xr egrep 164talks EREs.65Consult the manual page for the specific application to find out which66it uses.67.Pp68POSIX leaves some aspects of RE syntax and semantics open;69.Sq **70marks decisions on these aspects that71may not be fully portable to other POSIX implementations.72.Pp73This manual page first describes regular expressions in general,74specifically extended regular expressions,75and then discusses differences between them and basic regular expressions.76.Sh EXTENDED REGULAR EXPRESSIONS77An ERE is one** or more non-empty**78.Em branches ,79separated by80.Sq \*(Ba .81It matches anything that matches one of the branches.82.Pp83A branch is one** or more84.Em pieces ,85concatenated.86It matches a match for the first, followed by a match for the second, etc.87.Pp88A piece is an89.Em atom90possibly followed by a single**91.Sq * ,92.Sq + ,93.Sq ?\& ,94or95.Em bound .96An atom followed by97.Sq *98matches a sequence of 0 or more matches of the atom.99An atom followed by100.Sq +101matches a sequence of 1 or more matches of the atom.102An atom followed by103.Sq ?\&104matches a sequence of 0 or 1 matches of the atom.105.Pp106A bound is107.Sq {108followed by an unsigned decimal integer,109possibly followed by110.Sq ,\&111possibly followed by another unsigned decimal integer,112always followed by113.Sq } .114The integers must lie between 0 and115.Dv RE_DUP_MAX116(255**) inclusive,117and if there are two of them, the first may not exceed the second.118An atom followed by a bound containing one integer119.Ar i120and no comma matches121a sequence of exactly122.Ar i123matches of the atom.124An atom followed by a bound125containing one integer126.Ar i127and a comma matches128a sequence of129.Ar i130or more matches of the atom.131An atom followed by a bound132containing two integers133.Ar i134and135.Ar j136matches a sequence of137.Ar i138through139.Ar j140(inclusive) matches of the atom.141.Pp142An atom is a regular expression enclosed in143.Sq ()144(matching a part of the regular expression),145an empty set of146.Sq ()147(matching the null string)**,148a149.Em bracket expression150(see below),151.Sq .\&152(matching any single character),153.Sq ^154(matching the null string at the beginning of a line),155.Sq $156(matching the null string at the end of a line),157a158.Sq \e159followed by one of the characters160.Sq ^.[$()|*+?{\e161(matching that character taken as an ordinary character),162a163.Sq \e164followed by any other character**165(matching that character taken as an ordinary character,166as if the167.Sq \e168had not been present**),169or a single character with no other significance (matching that character).170A171.Sq {172followed by a character other than a digit is an ordinary character,173not the beginning of a bound**.174It is illegal to end an RE with175.Sq \e .176.Pp177A bracket expression is a list of characters enclosed in178.Sq [] .179It normally matches any single character from the list (but see below).180If the list begins with181.Sq ^ ,182it matches any single character183.Em not184from the rest of the list185(but see below).186If two characters in the list are separated by187.Sq - ,188this is shorthand for the full189.Em range190of characters between those two (inclusive) in the191collating sequence, e.g.\&192.Sq [0-9]193in ASCII matches any decimal digit.194It is illegal** for two ranges to share an endpoint, e.g.\&195.Sq a-c-e .196Ranges are very collating-sequence-dependent,197and portable programs should avoid relying on them.198.Pp199To include a literal200.Sq ]\&201in the list, make it the first character202(following a possible203.Sq ^ ) .204To include a literal205.Sq - ,206make it the first or last character,207or the second endpoint of a range.208To use a literal209.Sq -210as the first endpoint of a range,211enclose it in212.Sq [.213and214.Sq .]215to make it a collating element (see below).216With the exception of these and some combinations using217.Sq [218(see next paragraphs),219all other special characters, including220.Sq \e ,221lose their special significance within a bracket expression.222.Pp223Within a bracket expression, a collating element224(a character,225a multi-character sequence that collates as if it were a single character,226or a collating-sequence name for either)227enclosed in228.Sq [.229and230.Sq .]231stands for the sequence of characters of that collating element.232The sequence is a single element of the bracket expression's list.233A bracket expression containing a multi-character collating element234can thus match more than one character,235e.g. if the collating sequence includes a236.Sq ch237collating element,238then the RE239.Sq [[.ch.]]*c240matches the first five characters of241.Sq chchcc .242.Pp243Within a bracket expression, a collating element enclosed in244.Sq [=245and246.Sq =]247is an equivalence class, standing for the sequences of characters248of all collating elements equivalent to that one, including itself.249(If there are no other equivalent collating elements,250the treatment is as if the enclosing delimiters were251.Sq [.252and253.Sq .] . )254For example, if255.Sq x256and257.Sq y258are the members of an equivalence class,259then260.Sq [[=x=]] ,261.Sq [[=y=]] ,262and263.Sq [xy]264are all synonymous.265An equivalence class may not** be an endpoint of a range.266.Pp267Within a bracket expression, the name of a268.Em character class269enclosed270in271.Sq [:272and273.Sq :]274stands for the list of all characters belonging to that class.275Standard character class names are:276.Bd -literal -offset indent277alnum digit punct278alpha graph space279blank lower upper280cntrl print xdigit281.Ed282.Pp283These stand for the character classes defined in284.Xr ctype 3 .285A locale may provide others.286A character class may not be used as an endpoint of a range.287.Pp288There are two special cases** of bracket expressions:289the bracket expressions290.Sq [[:<:]]291and292.Sq [[:>:]]293match the null string at the beginning and end of a word, respectively.294A word is defined as a sequence of295characters starting and ending with a word character296which is neither preceded nor followed by297word characters.298A word character is an299.Em alnum300character (as defined by301.Xr ctype 3 )302or an underscore.303This is an extension,304compatible with but not specified by POSIX,305and should be used with306caution in software intended to be portable to other systems.307.Pp308In the event that an RE could match more than one substring of a given309string,310the RE matches the one starting earliest in the string.311If the RE could match more than one substring starting at that point,312it matches the longest.313Subexpressions also match the longest possible substrings, subject to314the constraint that the whole match be as long as possible,315with subexpressions starting earlier in the RE taking priority over316ones starting later.317Note that higher-level subexpressions thus take priority over318their lower-level component subexpressions.319.Pp320Match lengths are measured in characters, not collating elements.321A null string is considered longer than no match at all.322For example,323.Sq bb*324matches the three middle characters of325.Sq abbbc ;326.Sq (wee|week)(knights|nights)327matches all ten characters of328.Sq weeknights ;329when330.Sq (.*).*331is matched against332.Sq abc ,333the parenthesized subexpression matches all three characters;334and when335.Sq (a*)*336is matched against337.Sq bc ,338both the whole RE and the parenthesized subexpression match the null string.339.Pp340If case-independent matching is specified,341the effect is much as if all case distinctions had vanished from the342alphabet.343When an alphabetic that exists in multiple cases appears as an344ordinary character outside a bracket expression, it is effectively345transformed into a bracket expression containing both cases,346e.g.\&347.Sq x348becomes349.Sq [xX] .350When it appears inside a bracket expression,351all case counterparts of it are added to the bracket expression,352so that, for example,353.Sq [x]354becomes355.Sq [xX]356and357.Sq [^x]358becomes359.Sq [^xX] .360.Pp361No particular limit is imposed on the length of REs**.362Programs intended to be portable should not employ REs longer363than 256 bytes,364as an implementation can refuse to accept such REs and remain365POSIX-compliant.366.Pp367The following is a list of extended regular expressions:368.Bl -tag -width Ds369.It Ar c370Any character371.Ar c372not listed below matches itself.373.It \e Ns Ar c374Any backslash-escaped character375.Ar c376matches itself.377.It \&.378Matches any single character that is not a newline379.Pq Sq \en .380.It Bq Ar char-class381Matches any single character in382.Ar char-class .383To include a384.Ql \&]385in386.Ar char-class ,387it must be the first character.388A range of characters may be specified by separating the end characters389of the range with a390.Ql - ;391e.g.\&392.Ar a-z393specifies the lower case characters.394The following literal expressions can also be used in395.Ar char-class396to specify sets of characters:397.Bd -unfilled -offset indent398[:alnum:] [:cntrl:] [:lower:] [:space:]399[:alpha:] [:digit:] [:print:] [:upper:]400[:blank:] [:graph:] [:punct:] [:xdigit:]401.Ed402.Pp403If404.Ql -405appears as the first or last character of406.Ar char-class ,407then it matches itself.408All other characters in409.Ar char-class410match themselves.411.Pp412Patterns in413.Ar char-class414of the form415.Eo [.416.Ar col-elm417.Ec .]\&418or419.Eo [=420.Ar col-elm421.Ec =]\& ,422where423.Ar col-elm424is a collating element, are interpreted according to425.Xr setlocale 3426.Pq not currently supported .427.It Bq ^ Ns Ar char-class428Matches any single character, other than newline, not in429.Ar char-class .430.Ar char-class431is defined as above.432.It ^433If434.Sq ^435is the first character of a regular expression, then it436anchors the regular expression to the beginning of a line.437Otherwise, it matches itself.438.It $439If440.Sq $441is the last character of a regular expression,442it anchors the regular expression to the end of a line.443Otherwise, it matches itself.444.It [[:<:]]445Anchors the single character regular expression or subexpression446immediately following it to the beginning of a word.447.It [[:>:]]448Anchors the single character regular expression or subexpression449immediately following it to the end of a word.450.It Pq Ar re451Defines a subexpression452.Ar re .453Any set of characters enclosed in parentheses454matches whatever the set of characters without parentheses matches455(that is a long-winded way of saying the constructs456.Sq (re)457and458.Sq re459match identically).460.It *461Matches the single character regular expression or subexpression462immediately preceding it zero or more times.463If464.Sq *465is the first character of a regular expression or subexpression,466then it matches itself.467The468.Sq *469operator sometimes yields unexpected results.470For example, the regular expression471.Ar b*472matches the beginning of the string473.Qq abbb474(as opposed to the substring475.Qq bbb ) ,476since a null match is the only leftmost match.477.It +478Matches the singular character regular expression479or subexpression immediately preceding it480one or more times.481.It ?482Matches the singular character regular expression483or subexpression immediately preceding it4840 or 1 times.485.Sm off486.It Xo487.Pf { Ar n , m No }\ \&488.Pf { Ar n , No }\ \&489.Pf { Ar n No }490.Xc491.Sm on492Matches the single character regular expression or subexpression493immediately preceding it at least494.Ar n495and at most496.Ar m497times.498If499.Ar m500is omitted, then it matches at least501.Ar n502times.503If the comma is also omitted, then it matches exactly504.Ar n505times.506.It \*(Ba507Used to separate patterns.508For example,509the pattern510.Sq cat\*(Badog511matches either512.Sq cat513or514.Sq dog .515.El516.Sh BASIC REGULAR EXPRESSIONS517Basic regular expressions differ in several respects:518.Bl -bullet -offset 3n519.It520.Sq \*(Ba ,521.Sq + ,522and523.Sq ?\&524are ordinary characters and there is no equivalent525for their functionality.526.It527The delimiters for bounds are528.Sq \e{529and530.Sq \e} ,531with532.Sq {533and534.Sq }535by themselves ordinary characters.536.It537The parentheses for nested subexpressions are538.Sq \e(539and540.Sq \e) ,541with542.Sq (543and544.Sq )\&545by themselves ordinary characters.546.It547.Sq ^548is an ordinary character except at the beginning of the549RE or** the beginning of a parenthesized subexpression.550.It551.Sq $552is an ordinary character except at the end of the553RE or** the end of a parenthesized subexpression.554.It555.Sq *556is an ordinary character if it appears at the beginning of the557RE or the beginning of a parenthesized subexpression558(after a possible leading559.Sq ^ ) .560.It561Finally, there is one new type of atom, a562.Em back-reference :563.Sq \e564followed by a non-zero decimal digit565.Ar d566matches the same sequence of characters matched by the567.Ar d Ns th568parenthesized subexpression569(numbering subexpressions by the positions of their opening parentheses,570left to right),571so that, for example,572.Sq \e([bc]\e)\e1573matches574.Sq bb\&575or576.Sq cc577but not578.Sq bc .579.El580.Pp581The following is a list of basic regular expressions:582.Bl -tag -width Ds583.It Ar c584Any character585.Ar c586not listed below matches itself.587.It \e Ns Ar c588Any backslash-escaped character589.Ar c ,590except for591.Sq { ,592.Sq } ,593.Sq \&( ,594and595.Sq \&) ,596matches itself.597.It \&.598Matches any single character that is not a newline599.Pq Sq \en .600.It Bq Ar char-class601Matches any single character in602.Ar char-class .603To include a604.Ql \&]605in606.Ar char-class ,607it must be the first character.608A range of characters may be specified by separating the end characters609of the range with a610.Ql - ;611e.g.\&612.Ar a-z613specifies the lower case characters.614The following literal expressions can also be used in615.Ar char-class616to specify sets of characters:617.Bd -unfilled -offset indent618[:alnum:] [:cntrl:] [:lower:] [:space:]619[:alpha:] [:digit:] [:print:] [:upper:]620[:blank:] [:graph:] [:punct:] [:xdigit:]621.Ed622.Pp623If624.Ql -625appears as the first or last character of626.Ar char-class ,627then it matches itself.628All other characters in629.Ar char-class630match themselves.631.Pp632Patterns in633.Ar char-class634of the form635.Eo [.636.Ar col-elm637.Ec .]\&638or639.Eo [=640.Ar col-elm641.Ec =]\& ,642where643.Ar col-elm644is a collating element, are interpreted according to645.Xr setlocale 3646.Pq not currently supported .647.It Bq ^ Ns Ar char-class648Matches any single character, other than newline, not in649.Ar char-class .650.Ar char-class651is defined as above.652.It ^653If654.Sq ^655is the first character of a regular expression, then it656anchors the regular expression to the beginning of a line.657Otherwise, it matches itself.658.It $659If660.Sq $661is the last character of a regular expression,662it anchors the regular expression to the end of a line.663Otherwise, it matches itself.664.It [[:<:]]665Anchors the single character regular expression or subexpression666immediately following it to the beginning of a word.667.It [[:>:]]668Anchors the single character regular expression or subexpression669immediately following it to the end of a word.670.It \e( Ns Ar re Ns \e)671Defines a subexpression672.Ar re .673Subexpressions may be nested.674A subsequent backreference of the form675.Pf \e Ns Ar n ,676where677.Ar n678is a number in the range [1,9], expands to the text matched by the679.Ar n Ns th680subexpression.681For example, the regular expression682.Ar \e(.*\e)\e1683matches any string consisting of identical adjacent substrings.684Subexpressions are ordered relative to their left delimiter.685.It *686Matches the single character regular expression or subexpression687immediately preceding it zero or more times.688If689.Sq *690is the first character of a regular expression or subexpression,691then it matches itself.692The693.Sq *694operator sometimes yields unexpected results.695For example, the regular expression696.Ar b*697matches the beginning of the string698.Qq abbb699(as opposed to the substring700.Qq bbb ) ,701since a null match is the only leftmost match.702.Sm off703.It Xo704.Pf \e{ Ar n , m No \e}\ \&705.Pf \e{ Ar n , No \e}\ \&706.Pf \e{ Ar n No \e}707.Xc708.Sm on709Matches the single character regular expression or subexpression710immediately preceding it at least711.Ar n712and at most713.Ar m714times.715If716.Ar m717is omitted, then it matches at least718.Ar n719times.720If the comma is also omitted, then it matches exactly721.Ar n722times.723.El724.Sh SEE ALSO725.Xr ctype 3 ,726.Xr regex 3727.Sh STANDARDS728.St -p1003.1-2004 :729Base Definitions, Chapter 9 (Regular Expressions).730.Sh BUGS731Having two kinds of REs is a botch.732.Pp733The current POSIX spec says that734.Sq )\&735is an ordinary character in the absence of an unmatched736.Sq ( ;737this was an unintentional result of a wording error,738and change is likely.739Avoid relying on it.740.Pp741Back-references are a dreadful botch,742posing major problems for efficient implementations.743They are also somewhat vaguely defined744(does745.Sq a\e(\e(b\e)*\e2\e)*d746match747.Sq abbbd ? ) .748Avoid using them.749.Pp750POSIX's specification of case-independent matching is vague.751The752.Dq one case implies all cases753definition given above754is the current consensus among implementors as to the right interpretation.755.Pp756The syntax for word boundaries is incredibly ugly.757