Skip to main content

How to format huge Json files

Recently I had to deal with some huge JSON backup files and needed to do a simple text search. But the JSON file is compact and so huge like 5Mb. None of my text editors could handle that size of a compact JSON file. The thing is when the JSON is in a compact, it's just one line. So when I open it up with a text editor or an IDE, it just crashes. It can't handle that much of a lengthy line.




I could run a grep command in the command line but since it is just a single line, its not really useful to know the line number.

The trick is to format the JSON file with tabbed indentation, but online Json formatters weren't good enough for this task. same as the text editor it just crashes. Or they don't really allow you to format huge JSON files.

The answer

The answer is Python!

Python comes really handy at this work. Python comes with built-in JSON manipulation libraries, and you can run a python function in the command line even if you don't know how to code in python.

Say my JSON file is test.json. So simply run the below command.

python -m json.tool test.json > formatted.json

note that -m is to run the particular function in the command line. test.json is the input file. The greater than ">" allows you to write the output of the python function into a file.


Note that the above command is for Linux or Mac machines. For windows, this might be a little different.


Comments

Popular posts from this blog

Java, how to create a list with a single element

 I wanted to create a Java List with a single element. Yet, I wanted to add more elements later. So, I was looking for a couple of ways to do this. So far there are multiple elegant ways to create a list with a single element with a one-liner. Not so much for a modifiable list though. Here's what I gathered so far. Followings are a few ways of creating a list with strictly a single entry. Can't add more elements. 1. Collections.singletonList() This returns an immutable list that cannot be modified or add more elements. // An immutable list containing only the specified object. List<String> oneEntryList = Collections. singletonList ( "one" ) ; oneEntryList.add( "two" ) ; // throws UnsupportedOperationException 2.  Arrays.asList() This returns a fixed-size list consisting of the number of elements provided as arguments. The number of elements provided would be an array hence the size is fixed to the length of the array. // Returns a fixed-size list List...

Ubuntu DNS issue fix DNS_PROBE_FINISHED_BAD_CONFIG

Issue  I've been playing with a VPN and somehow it messed up my DNS resolution configurations. Chrome gives  DNS_PROBE_FINISHED_BAD_CONFIG  error and can't ping google. So it seemed to be an issue with the DNS. Of course, restarting didn't fix it. I tried DNS lookup which gave me below. To make sure this is somehting to do with my DNS confgis, I ran the same by providing the google DNS servers.  It works, which means my default DNS is not working for some reason. To make sure this, ran the below command. systemd-resolve --status Output has an entry for DNS Servers, which was  ::1 Fix 1. Edit the file /etc/systemd/resolved.conf. sudo vi /etc/systemd/resolved.conf 2. Add new DNS entries. I added 2 google DNS and the cloudflare DNS sever. [Resolve] DNS=8.8.8.8 8.8.4.4 1.1.1.1 3. Restart the systemd-resolved and check the configuration is persisted in /run/systemd/resolve/resolv.conf file. sudo service systemd-resolved restart cat /run/systemd/resolve/resolv.co...

Install Docker on Windows 11 with WSL Ubuntu 22.04

This is to install Docker within Ubuntu WSL without using the Windows Docker application. Follow the below steps. Install Ubuntu 22.04 WSL 1. Enable Windows Subsystem for Linux and Virtual Machine platform Go to Control Panel -> Programs -> Programs and Features -> Turn Windows features on or off 2. Switch to WSL 2 Open Powershell and type in the below command. wsl --set-default-version 2 If you don't have WSL 2, download the latest WSL 2 package and install it.  3. Install Ubuntu Open Microsoft Store and search for Ubuntu. Select the version you intend to install. I'd use the latest LTS version Ubuntu 22.04. Click on the Get button. It will take a couple of minutes to download and install. 4. Open up the installed Ubuntu version that was installed. If you get an error like the below image, make sure to install the WSL2 Kernel update .  If it's an older Ubuntu version the error message would be something like the image below. Error: WSL 2 requires an update to its ...