Using the strtok() and strtok_r() functions in C With Examples

In this article, we’ll take a look at using the strtok() and strtok_r() functions in C.

These functions are very useful, if you want to tokenize a string. C provides these handy utility functions to split our input string into tokens.

Let’s take a look at using these functions, using suitable examples.


Using the strtok() function

First, let’s look at the strtok() function.

This function is a part of the <string.h> header file, so you must include it in your program.

This takes in an input string str and a delimiter character delim.

strtok() will split the string into tokens based on the delimited character.

We expect a list of strings from strtok(). But the function returns us a single string! Why is this?

The reason is how the function handles the tokenization. After calling strtok(input, delim), it returns the first token.

But we must keep calling the function again and again on a NULL input string, until we get NULL!

Basically, we need to keep calling strtok(NULL, delim) until it returns NULL.

Seems confusing? Let’s look at an example to clear it out!

So, we have our input string “Hello from JournalDev!”, and we’re trying to tokenize it by spaces.

We get the first token using strtok(input, " "). Notice the double quotes, as the delimiter is a single character string!

Afterwards, we keep getting tokens using strtok(NULL, " ") and loop until we get NULL from strtok().

Let’s look at the output now.

Output

Indeed, we seem to have got the correct tokens!

Similarly, let’s now look at using strtok_r().


Using the strtok_r() function

This function is very similar to the strtok() function. The key difference is that the _r means that this is a re-entrant function.

A reentrant function is a function that can be interrupted during its execution. This type of function can also be safely called again, to resume execution!

This is why it is a “re-entrant” function. Just because it can safely enter again!

Due to this fact, re-entrant functions are thread-safe, meaning that they can safely be interrupted by threads, just because they can resume again without any harm.

Now, similar to strtok(), the strtok_r() function is a thread-safe version of it.

However, this has an extra parameter to it, called the context. We need this, so that the function can resume from the right place.

NOTE: If you’re using Windows, the equivalent function is strtok_s(). strtok_r() is for Linux / Mac based systems!

The context parameter is a pointer to the character, which strtok_r uses internally to save its state.

Usually, we can just pass it from a user-declared pointer.

Let’s look at the same example for strtok(), now using strtok_r() (or strtok_s() on Windows).

Output

While we get the same output, this version is better, since it is thread safe!


Conclusion

In this article, we learned about how we could use the strtok() and strtok_r() functions in C, to tokenize strings easily.

For similar content, do go through our tutorial section on C programming!

References


By admin

Leave a Reply