How to Use the merge-source-map Library

Greetings, friends! This article is a continuation of my post about the Magic String library which you can find here. Today, I'd like to discuss how to combine source maps together using the merge-source-map library. I will use the Magic String library to create source maps from small changes in our code and then proceed to use the merge-source-map library to merge the source maps together. Let's get started!

First, we'll create a new directory and navigate to it:

mkdir merge-source-map-library-tutorial && cd $_

Then, we'll bootstrap a new project with a package.json file:

npm init -y

We will install two packages:

npm i magic-string merge-source-map

Then, we will create an input.txt file containing the following contents:

😃 😃 😃

Alright. You're probably wondering why I created a .txt file and put three smiling emojis inside it. Well, you're in for a treat! 😃 We will be creating a small DSL that converts emojis into JavaScript code using what we learned in my previous tutorial on the Magic String library. Then, we will generate a source map that links back to the original input.txt file. You may think this is impossible, but I will show you that it can be done indeed. Source maps declare the content and filenames of whatever you want. As long as the source maps contain the original source code, it doesn't matter if the file is a .txt or .js file. It only cares about the content inside them.

To prove what I'm saying is true, we'll create a file called transformer-1.js that will take the smiling emojis and covert them to a JavaScript file called output-1.js.

const fs = require('fs');
const MagicString = require('magic-string');

const code = fs
  .readFileSync('input.txt', 'utf8')
  .replace(/\s/g,'');

const s = new MagicString(code);

const pattern = /😃/g;

let match;

while ((match = pattern.exec(code))) {
  const start = match.index;
  const end = start + match[0].length;
  s.overwrite(start, end, `console.log('happy');\n`);
}

const map = s.generateMap({
  source: 'input.txt',
  file: 'output-1.js.map',
  includeContent: true,
});

fs.writeFileSync(
  'output-1.js',
  s.toString() + '\n//# sourceMappingURL=' + 'output-1.js.map'
);
fs.writeFileSync('output-1.js.map', map.toString());

console.log('Generated output-1.js!');
console.log('Generated output-1.js.map!');

Execute this script by running node transformer-1 or node transformer-1.js.

This code will generate two files. The first file will be a JavaScript file, output-1.js, containing the following contents:

console.log('happy');
console.log('happy');
console.log('happy');

//# sourceMappingURL=output-1.js.map

The second file will be the corresponding source map, output-1.js.map, that contains the following contents:

{
  "version": 3,
  "file": "output-1.js.map",
  "sources": [
    "input.txt"
  ],
  "sourcesContent": [
    "😃😃😃"
  ],
  "names": [],
  "mappings": "AAAA;AAAE;AAAE;"
}

Notice that the smiling emojis are inside the sourcesContent property. This source map contains a mapping between the code in output.js and the content of input.txt, and that's important to recognize. It's not a mapping between the file, input.txt, and output.js. It's a mapping between "😃😃😃" and output.js. The name you see in the sources property will be the name visible to us in Chrome Developer Tools when it finds our source maps. We could have called it anything we want, but I kept it as input.txt. If you use Webpack, then you'll likely see notation such as webpack:// in source maps Webpack generates, but that's just a naming convention the Webpack team uses.

You may also notice that the spaces between the smiling emojis are gone because our transformer-1.js script removed them using a regular expression:

const code = fs
  .readFileSync('input.txt', 'utf8')
  .replace(/\s/g,'');

We have created one source map now, but we need to create a second one. Let's create a second transformer named transformer-2.js that will modify the console.log statements inside output-1.js to include "super happy" instead of just "happy".

const fs = require('fs');
const MagicString = require('magic-string');

const code = fs
  .readFileSync('output-1.js', 'utf8')
  .replace(/\/\/# sourceMappingURL=output-1.js.map/g, '');

const s = new MagicString(code);

const pattern = /happy/g;

let match;

while ((match = pattern.exec(code))) {
  const start = match.index;
  const end = start + match[0].length;
  s.overwrite(start, end, `super happy`);
}

const map = s.generateMap({
  source: 'transformed-1.js',
  file: 'output-2.js.map',
  includeContent: true,
});

fs.writeFileSync(
  'output-2.js',
  s.toString() + '//# sourceMappingURL=' + 'output-2.js.map'
);
fs.writeFileSync('output-2.js.map', map.toString());

console.log('Generated output-2.js!');
console.log('Generated output-2.js.map!');

Execute this script by running node transformer-2 or node transformer-2.js.

This code will also generate two files. The first file will be a JavaScript file, output-2.js, containing the following contents:

console.log('super happy');
console.log('super happy');
console.log('super happy');

//# sourceMappingURL=output-2.js.map

The second file we be another source map called output-2.js.map containing the following contents:

{
  "version": 3,
  "file": "output-2.js.map",
  "sources": [
    "transformed-1.js"
  ],
  "sourcesContent": [
    "console.log('happy');\nconsole.log('happy');\nconsole.log('happy');\n\n"
  ],
  "names": [],
  "mappings": "AAAA,aAAa,WAAK;AAClB,aAAa,WAAK;AAClB,aAAa,WAAK;AAClB;"
}

We have now transformed the original input.txt code twice. Notice that this source map contains the contents of output-1.js, but not the contents of input.txt. Where did the smiling emojis go? 😲 The Magic String library was told to transform the contents of output-1.js and create a source map based on this transformation. However, it lost the information from the original source code found in input.txt. This is why we need to leverage another library: merge-source-map.

We will create a script that combines the source maps, output-1.js.map and output-2.js.map together into a single source map called merged.js.map. Let's call this script merge-maps.js:

const fs = require('fs');
const merge = require('merge-source-map');

const oldMap = fs.readFileSync('output-1.js.map');
const newMap = fs.readFileSync('output-2.js.map');

const mergedMap = merge(JSON.parse(oldMap), JSON.parse(newMap));

fs.writeFileSync('merged.js.map', JSON.stringify(mergedMap));

const modified = fs
  .readFileSync('output-2.js')
  .toString()
  .replace(/output-2/g, 'merged');

fs.writeFileSync('output-2.js', modified);

console.log('Generated merged.js.map!');

This code is rather simple. It reads the contents of both source maps and then we run it through a merge method provided by the merge-source-map library. Then, we take the contents of this merged map and save it to a file called merged.js.map. Since source maps contain JSON data, we can use JSON.parse to parse the source map. We then use JSON.stringify to ensure the merged source map is also configured to be in the JSON format.

There's one more step we did to make sure that we can use our new merged source map.

const modified = fs
  .readFileSync('output-2.js')
  .toString()
  .replace(/output-2/g, 'merged');

fs.writeFileSync('output-2.js', modified);

We overwrote this line in output-2.js:

//# sourceMappingURL=output-2.js.map

To be the following instead:

//# sourceMappingURL=merged.js.map

Now the output-2.js file links to a combined source map that contains the contents of both input.txt and output-1.js as seen in merged.js.map:

{
  "version": 3,
  "sources": [
    "input.txt",
    "transformed-1.js"
  ],
  "names": [],
  "mappings": "AAAA,aAAA,WAAA;AAAE,aAAA,WAAA;AAAE,aAAA,WAAA",
  "file": "output-1.js.map",
  "sourcesContent": [
    "😃😃😃",
    "console.log('happy');\nconsole.log('happy');\nconsole.log('happy');\n\n"
  ]
}

Pretty cool! 🎉

Let's see this merged source map in action! 🎬 We will create a simple index.html file with the following contents:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Merge Map Source Map Test</title>
</head>
<body>
  <h1>Merge Map Source Map Test</h1>
  <script src="output-2.js"></script>
</body>
</html>

Notice that we're using output-2.js that references the source map, merged.js.map. Let's install a small package to serve our assets:

npm i -D serve

Then, we can add the following script to our package.json file:

"start": "serve ."

When we run the command, npm start, we should be able to navigate to localhost:5000 (or whatever port it tells you) to see our code running.

If we open up the developer tools in Google Chrome, we should be able to navigate to output-2.js and insert breakpoints. If you activate the debugger, you should be able to map the console.log statements back to the original smiling emojis! 😃 😃 😃

Isn't that incredible! 🎉

Not only did you create a small DSL, but you also transformed your original code twice and created a source map that links the modified code together with the original code!!!

You can find the completed code here on my GitHub page. Please follow the instructions in this article or in the README.md file found in my GitHub repo to generate the output files and source maps.

Conclusion

I hope you learned a lot about source maps in this article! Using the Magic String library together with the merge-source-map can create some interesting applications! In fact, vite utilizes both of these tools to create an awesome build tool without the need for Webpack! Sometimes, you can learn a lot by just diving into the package.json of some popular libraries and seeing what tools were used. I hope you create amazing applications with these tools! 😃 😃 😃