Regular Expressions (sometimes called regex for short) allow a user to search for strings using almost any sort of rule they can come up with. For example, finding all capital letters in a string, or finding a phone number in a document.
Regular expressions are notorious for their seemingly strange syntax. This strange syntax is a byproduct of their flexibility. Regular expressions have to be able to filter out any string pattern you can imagine, which is why they have a complex string pattern format.
Regular expressions are handled using Python's built-in re library. See the docs for more information.
Let's begin by explaining how to search for basic patterns in a string!
Searching for Basic Patterns
Let's imagine that we have the following string:
text = "The agent's phone number is 408-555-1234. Call soon!"
We'll start off by trying to find out if the string "phone" is inside the text string. Now we could quickly do this with:
'phone' in text return true
But let's show the format for regular expressions, because later on we will be searching for patterns that won't have such a simple solution.
import re pattern = 'phone' match = re.search(pattern,text)
Now we've seen that re.search() will take the pattern, scan the text, and then returns a Match object. If no pattern is found, a None is returned (in Jupyter Notebook this just means that nothing is output below the cell).
Let's take a closer look at this Match object.
Notice the span, there is also a start and end index information.
match.span() //(12,17) match.start() //12 match.end() //17
But what if the pattern occurs more than once?
text = "my phone is a new phone" match = re.search(pattern,text) match.span() //(3,8)
Notice it only matches the first instance. If we wanted a list of all matches, we can use .findall() method:
matches = re.findall("phone",text)
So far we've learned how to search for a basic string. What about more complex examples? Such as trying to find a telephone number in a large string of text? Or an email address?
We could just use search method if we know the exact phone or email, but what if we don't know it? We may know the general format, and we can use that along with regular expressions to search the document for strings that match a particular pattern.
This is where the syntax may appear strange at first, but take your time with this; often it's just a matter of looking up the pattern code.
Identifiers for Characters in Patterns
Characters such as a digit or a single string have different codes that represent them. You can use these to build up a pattern string. Notice how these make heavy use of the backwards slash \ . Because of this when defining a pattern string for regular expression we use the format:
placing the r in front of the string allows python to understand that the \ in the pattern string are not meant to be escape slashes.
Now that we know the special character designations, we can use them along with quantifiers to define how many we expect.
What if we wanted to do two tasks, find phone numbers, but also be able to quickly extract their area code (the first three digits). We can use groups for any general task that involves grouping together regular expressions (so that we can later break them down).
Using the phone number example, we can separate groups of regular expressions using parentheses: