JSON Parser with ANTLR4 and Eclipse

In this tutorial we will create a JSON Parser using ANTLR4 and Eclipse.

1. Prerequisites

  1. EclipseEE (for Project Facets settings)
  2. ANTLR4 IDE Eclipse plugin
  3. ANTLR4 JAR file which you can download here: antlr-4.4-complete.jar

2. Install ANTLR4 Eclipse plugin

  1. In Eclipse go to Help  Eclipse Marketplace
  2. Search for “ANTLR4” and install it (Current version: ANTLR 4 IDE 0.3.5)
  3. Restart Eclipse

3. Create ANTLR4 Project

  1. Right click in the Project Explorer and select New Project
  2. Search for “ANTLR” and select ANTLR 4 Project
  3. Name the project e.g. “jsonParser”

4. Adapt the ANTLR4 Project

The project wizard creates a simple hello.g4 grammar and automatically builds Lexer, Parser etc.

ANTLR4 Project Explorer
ANTLR4 project structure

In order to program we have to adapt the Project Facets and add the generated files to our source folder as well as add the ANTLR4 JAR file to the Java Build Path.

  1. Adapt the Project Facets
    1. Right click your created project and click on properties
    2. Select Project Facets and click on “Convert to faceted form…”
    3. Check Java and adapt he version
    4. Click Apply and OK

      ANTLR4 Project Facets
      ANTLR4 Project Facets
  2. Add ANTLR4 target to the Java source folder
    1. Right click your created project and click on properties
    2. Select “Java Build Path” and click on “Add Folder” in the Source tab
    3. Select the “target/generated-sources/antlr4” folder and  press OK
    4. You will see some errors because you have to add the ANTLR4 JAR file to your project first
  3. Recommended (path independent): Add the ANTLR4 JAR to your project (download and copy)
    1. Create a new folder called “lib” in your project root folder
    2. Copy the antlr-4.4-complete.jar into “lib”
    3. Right click the JAR file and select Build Path ⇒ Add to Build Path
    4. The errors should be resolved
  4. Optional (path dependent): Add the ANTLR4 JAR to your project (from Eclipse plugin)
    1. Right click your project and select Properties
    2. Select Java Build Path and click on the tab Libraries
    3. Click on “Add external JARs”
    4. The path from the library differs from the OS you are using. Check your console output where the ANTLR4 Tool logs:

Depending whether you only develop on one computer, you can select option 4. We recommand downloading the JAR though. If you use another version than the ANTLR Tool, you may receive a warning in the console later when running the program.

5. ANTLR4 JSON Grammar

We use a predefined grammar created by Terence Parr from json.org. Grammars are usually represented in the Backus-Naur Form (BNF). If you have some basic knowledge about e.g. grammars or compilers, the following grammar should be understandable. Otherwise we give some help and explanation below. Please replace the content of hello.g4 in your root folder with the following grammar and rename the file into JSON.g4.

ANTLR4 will automatically generate the necessary Lexer, Tokens and Parser. You can remove all files with the “Hello” Prefix from the target folder.

6. Run a small JSON example

In order to test the grammar with a JSON file, create a new Java class in the default package. We call it JSONParserTest.

This program reads a JSON file and uses the parser and lexer created from the ANTLR Tool to analyze the JSON file. If you do not want to read a file you can simply skip the first lines from the main method and replace the “content” variable with a JSON string of your own. The jsonExample.txt file looks like the following and should be in the “target/generated-sources/antlr4” folder.

Now you can right click the main class (in our case JSONParserTest) and select Run As ⇒ Java Application. You should receive a console output like this:

This represents the parse tree that is created from ANTLR. It may look confusing, but you can compare it to the JSON.g4 grammar and recognize keywords like object, pair, value etc.

You can now try to add errors to the JSON example file and run the program again. You will see that ANTLR does not recognize the wrong tokens. If the parser does not expect a certain character or sth. like that, you can abort and print out the faulty line.

7. Improve the output

You can use a nice Java class from Bart Kiers to improve the output from the parse tree. Use this AST class and add it to the “target/generated-sources/antlr4” folder:

Finally add the following code lines at the bottom of your main class:

Now you can check the output and see exactly which tokens are recognized and to which type (defined in JSON.tokens) the tokens are assigned.

8. Conclusion

ANTLR4 is a very powerful tool and supports you with writing lexers and parsers. Please search for a some easy example grammars and run them with the code above.

In order to understand how to access the tokens / objects, please study the AST class. Furthermore, this will teach you how to integrate these tools into your own applications.

If you have questions or problems, feel free to ask or comment.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

3 Thoughts to “JSON Parser with ANTLR4 and Eclipse”

  1. Hi,

    I have to say this tutorial is amazing.

    Thanks a ton,
    Mitali

  2. menzellu

    Hi, thanks for publishing this blog, looks really interesting. I am learning about ANTLR, would be good to try this out. I am using eclipse Mars, after creating the ANTLR 4 project, I opened the project properties, but I don’t see a section called “Project Facets” …. please help.

    1. Hi menzellu,

      do you mean the Project Facets option doesnt appear at all?
      Depending on the project you selected, you probably have to click the “Convert to faceted form” link on the right hand sind of the properties menu.

      I never had that problem (using Mars as well) before.

      Maybe try to recreate the project or even install a fresh copy of eclipse to make sure you didnt mess with any eclipse properties.

      I hope this helps,
      Malte

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.