Skip to main content

How to format huge Json files

Recently I had to deal with some huge JSON backup files and needed to do a simple text search. But the JSON file is compact and so huge like 5Mb. None of my text editors could handle that size of a compact JSON file. The thing is when the JSON is in a compact, it's just one line. So when I open it up with a text editor or an IDE, it just crashes. It can't handle that much of a lengthy line.




I could run a grep command in the command line but since it is just a single line, its not really useful to know the line number.

The trick is to format the JSON file with tabbed indentation, but online Json formatters weren't good enough for this task. same as the text editor it just crashes. Or they don't really allow you to format huge JSON files.

The answer

The answer is Python!

Python comes really handy at this work. Python comes with built-in JSON manipulation libraries, and you can run a python function in the command line even if you don't know how to code in python.

Say my JSON file is test.json. So simply run the below command.

python -m json.tool test.json > formatted.json

note that -m is to run the particular function in the command line. test.json is the input file. The greater than ">" allows you to write the output of the python function into a file.


Note that the above command is for Linux or Mac machines. For windows, this might be a little different.


Comments

Popular posts from this blog

Install Docker on Windows 11 with WSL Ubuntu 22.04

This is to install Docker within Ubuntu WSL without using the Windows Docker application. Follow the below steps. Install Ubuntu 22.04 WSL 1. Enable Windows Subsystem for Linux and Virtual Machine platform Go to Control Panel -> Programs -> Programs and Features -> Turn Windows features on or off 2. Switch to WSL 2 Open Powershell and type in the below command. wsl --set-default-version 2 If you don't have WSL 2, download the latest WSL 2 package and install it.  3. Install Ubuntu Open Microsoft Store and search for Ubuntu. Select the version you intend to install. I'd use the latest LTS version Ubuntu 22.04. Click on the Get button. It will take a couple of minutes to download and install. 4. Open up the installed Ubuntu version that was installed. If you get an error like the below image, make sure to install the WSL2 Kernel update .  If it's an older Ubuntu version the error message would be something like the image below. Error: WSL 2 requires an update to its ...

Java, how to create a list with a single element

 I wanted to create a Java List with a single element. Yet, I wanted to add more elements later. So, I was looking for a couple of ways to do this. So far there are multiple elegant ways to create a list with a single element with a one-liner. Not so much for a modifiable list though. Here's what I gathered so far. Followings are a few ways of creating a list with strictly a single entry. Can't add more elements. 1. Collections.singletonList() This returns an immutable list that cannot be modified or add more elements. // An immutable list containing only the specified object. List<String> oneEntryList = Collections. singletonList ( "one" ) ; oneEntryList.add( "two" ) ; // throws UnsupportedOperationException 2.  Arrays.asList() This returns a fixed-size list consisting of the number of elements provided as arguments. The number of elements provided would be an array hence the size is fixed to the length of the array. // Returns a fixed-size list List...

How to fix SSLHandshakeException PKIX path building failed in Java

TL ; DR 1. Extract the public certificate of the website/API that you are trying to connect from your Java application. Steps are mentioned in this post 2. Use the Java keytool to install the extracted certificate into the "cacerts" file (Trust store) keytool -import -trustcacerts -alias <domain name> -file <public certificate>.cert -keystore /path_to_java_home/jre/lib/security/cacerts -storepass changeit 3. Restart your Java application Exception A typical exception stack trace would look like below. javax.net.ssl. SSLHandshakeException : sun.security.validator.ValidatorException: PKIX path building failed : sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) at sun.security.ssl.Handshake...