Tokens in C Programming – Complete Guide to Top 7 Types with Examples

In this article, we will explore the fundamental concept of tokens in the C language. Tokens are the smallest unit in the C programming language and play a pivotal role in every C program. Understanding them is key to mastering C programming. Let’s explore tokens in C.

What are Tokens in C?

Token is the smallest unit in the C language. It is used for building blocks used to construct or develop together a C program. These tokens are very meaningful to the C compiler. C compiler breaks a program into small units, which are called tokens.

Program

Output

Compilation Process: C Program → Group Characters into C tokens —> Translate tokens into target

In this section, we will discuss some token rules:

• In C language, Tokens are building blocks, which means we cannot create a software program without using tokens.
• Tokens are classified into various subcategories. These are the main categories of identifiers, keywords, and operators. Tokens can be used to perform various operations, etc.

Types of Tokens in C language

In C programming language, Tokens can be classified into six types based on the functions they are used to perform. The types of C tokens are as follows:

• Keywords
• Identifiers
• Constants
• Strings
• Operators

Keywords

In the C language, Keywords are a collection of pre-defined or reserved words. They are case-sensitive and written in lowercase. The compiler already knows their meaning. We cannot use these keywords as variable names or function names because doing so would try to assign a new meaning to the keyword, which is not allowed in the C programming language. The C language supports 32 keywords.

The following table shows all the C keywords:

Example

Here, we will use the int, char, and auto keywords, which will automatically identify the data type for storage purposes.

The following program demonstrates the Keywords:

Program

Output

Identifiers

Identifiers are used for naming the variables, functions, and arrays. These are user-defined names consisting of an arbitrarily long response of letters and digits with either a letter or an underscore(_) as the first character. In C language, we cannot use C keywords as an identifier. They are reserved for special use cases. After declaring the identifiers. You can use the identifier in later program statements.

Rules for Naming Identifiers

There are certain rules followed before C declares the identifiers:

• They must begin with a letter or underscore_
• The identifier must consist of only letters, digits, or underscores. Identifying other special characters is not allowed.
• It should not be a keyword.
• It did not contain white space.
• It should be up to 31 characters long, as only the first 31 characters are significant.

Note: Identifiers are case-sensitive so names like variable and Variable will be treated as different.

For Example

• main: Method name
• A: variable name

Some valid identifiers, example

Test, _test,  test123, test_123, test1_ , Test

The above example shows the valid identifiers. These examples followed the identifiers’ ruling names. They do not start with a numerical digit or a keyword, and also, there is no blank space between or any special character.

Some invalid identifiers:

100test

_hello, morning

Int

Float

Kashmir(230)

The above examples do not follow all the essential rules for defining identifiers, so these names are not valid identifiers.

Constants

In C language, constants refer to variables with fixed values. They are not like normal variables. There is a difference between normal variables and constant values. Once constants are defined, they can not be modified in the program.

Constants may belong to any of the data types in the C language.

Examples of Constants in C

Types of Constants in C language

Strings

In C language, Strings are nothing but an array of characters ending with a null character (‘0’).  It indicates the null character end of the string. String are always enclosed in the double quotes. Whereas a character is enclosed in single quotes in C and C++ language.

Examples of String

Operators

In C language, Operators are symbols that trigger an action when applied to C variables and other objects. We can use operators with any kind of operand.  Let’s understand all type of operators in the C language.

• Unary Operators: These types of operators only require a single operand for performing some operations. Examples are the increment and decrement operators in the C language.
• Binary Operators: Binary operators require two operands to act upon are called binary operators. Binary Operators can be classified into:

1. Arithmetic Operators
2. Relation Operators
3. Logical Operators
4. Assignment Operators
5. Bitwise Operators

• Ternary Operator: This operator requires three operands to act upon and is called the ternary operator. The conditional operator(?) is also called the ternary operator.

The following program demonstrates the Operators example:

Program

Output

Special Characters in C

There are some special symbols used in the C language. Those symbols have some special meaning. It cannot be used for some other purposes. Let’s see those symbols.

• Brackets[]: Opening and closing brackets are used for defining the array in C language. This indicates the single and multidimensional subscripts.
• Parentheses(): This symbol is used for function calls and function parameters.
• Braces{}: These opening and ending curly braces mark the start and end of a block of code in C language. It can contain one or more statements.
• Comma(,): This character separates one more element in the array or defines multiple variables in a single line.
• Colon(:): It is an operator that essentially invokes something called an initialization list.
• Semicolon(;): It is known as a statement terminator. It is used for terminating one statement that must be ended with a semicolon.
• Asterisk(*): It is used for a pointer variable and for the multiplication of two operands or variables.
• Assignment Operator(=): It is used to assign values and for logical operation validation.
• Pre-processor(#): It is a macro processor that is used automatically by the compiler to transform your program before actual compilation.
• Period(.): It is used to access members of a structure or union.
• Tilde(~): It is the one’s complement operator.

Example to Implement C Tokens

In this section, we will implement Tokens in the C programming language. In this C program, we will use the different types of keywords available in the C language, such as float, char, const, short, and return.

The following program demonstrates the Tokens in C language:

Program

Output

Conclusion

In this article, we learned about Tokens in the C programming language. Tokens are the building blocks in the C programming language. They are the smallest unit that the compiler recognizes. We learned all types of tokens in the C language. Understanding these token types is crucial for writing, understanding, and debugging C programs effectively. The developer must be a master in tokens for writing efficient code in C.