Extracting and Publishing the Circle of Fifths Component

Tuesday, April 25th, 2023

Contents

• Introduction
• The Lay of the Land
• Enter The Git-Fu
• Preparing Circle Of Fifth's New Repo
• Preparing react-circle-of-fifths for Publication
• Publishing react-circle-of-fifths to NPM
• Integrating Our New Package
• Closing Thoughts

Introduction

Alright, so, I've been building my Circle of Fifths component, and it's getting to the point where I think I'm ready to start working on the actual functionality of the Music Theory application that I initially set out to build^[1]. But first, we're going to take a little digression.

As I was first starting to play around with the idea of Practicker, I took some time to search around and see if there were any existing Circle of Fifths components out there. If you've been reading this series up until now, it may not surprise you to learn that I didn't find anything that I liked. So, I decided I'd like to take all this work I've been doing and publish a publicly available React component. Today, we'll take a look at the process of extracting our component from the original repository and publishing it to NPM. Let's get started!

The Lay of the Land

Let's take a quick look at the layout of my project:

❯ tree -I 'node_modules|dist'
.
├── index.html
├── package.json
└── projects
    ├── key-thing
    │   ├── package.json
    │   ├── src
    │   │   ├── App.css
    │   │   ├── App.tsx
    │   │   ├── CircleOfFifths.css
    │   │   ├── CircleOfFifths.tsx
    │   │   ├── index.html
    │   │   ├── index.js
    │   │   └── main.css
    │   ├── tailwind.config.js
    │   └── yarn.lock
    └── template
        ├── package.json
        ├── src
        │   ├── App.tsx
        │   ├── index.html
        │   └── index.js
        ├── tailwind.config.js
        └── yarn.lock

5 directories, 19 files
 

So, there's a few different things in this directory. This project was originally set up as a place to play with several different ideas. I've owned the practicker.com domain for a while now, and I've been planning to stick some musically inclined practice tools there. My original thinking when setting up this repo was that I could set up a static HTML landing page at the top level, and link to a few different little projects. The template directory you see above is a skeleton project for each of these other little tools, and key-thing is the page / app that will eventually host my "Music Theory Dashboard".

Now, the simple and sensible way to extract out the component would be:

1. mv projects/key-thing/src/CircleOfFifths* new-directory
2. Continue working

Enter the Git-Fu

After a few minutes of Googling, I figure out an approach to this ill-advised idea. It looks like we can use the git filter-branch command to find the relevant commits and push them into another repository. This StackOverflow Link was really useful as I worked through this.

It looks like the basic idea here is this:

• Use filter-branch to create a branch containing only the commits that have touched the files I care about. In this case, that's only two files - CircleOfFifths.tsx and CircleOfFifths.css.
• Make a new, empty repository for the component.
• Push the branch we got in step 1 to the new repo.

Because I'm at least smart enough to know when I'm playing with fire, the very first thing I do is to make a backup copy of my repository. This is generally good practice when you decide to get frisky with Git, as it means you can mess up without fear of actually losing any history. We'll do our Git voodoo in here.

cp -r practicker-tools practicker-tools-extract
    

Alright, now let's give this a whirl.

git filter-branch --subdirectory-filter projects/key-thing  -- --all
    

This gives us a scary warning, and after lingering on the screen just long enough for you to start second-guessing yourself, proceeds to do what we asked for:

WARNING: git-filter-branch has a glut of gotchas generating mangled history
rewrites.  Hit Ctrl-C before proceeding to abort, then use an
alternative filtering tool such as 'git filter-repo'
(https://github.com/newren/git-filter-repo/) instead.  See the
filter-branch manual page for more details; to squelch this warning,
set FILTER_BRANCH_SQUELCH_WARNING=1.

Proceeding with filter-branch...

Rewrite 805f03e0d0f0d77f084f3b38c64fe3068319f1d7 (9/18) (1 seconds passed, remaining 1 predicted)
Ref 'refs/heads/master' was rewritten
Ref 'refs/stash' was rewritten

This leaves behind a bunch of files, which are now considered to be untracked by Git. We'll clean those up with a quick git clean -fdx. Now, if we take a look at our history, we can see that we're on the right track; now we have just the commits that touched files in the key-thing directory. Observe, my creative process laid bare:

~/src/practicker-tools-extract master 56s
❯ git log --oneline
87301cd (HEAD -> master) use sharp/flat symbols
4698c31 interactivity sample
d3d797e click handling
937b57d style hover state
26b4397 before adding interaction
4895cd6 reorder data
c2f435c lookin nice
c8376d2 properly positioned text
7b258c7 initial wedge layout
1003bbc wedges!
69e7fde ok, so breaking it into a new component works
d2778d9 why does the function call version not work
1e6b397 functioning hardcoded path
c9e9521 ok... why doesn't work
2b069fe viewbox
492c036 basic
f9027ca init key-thing

But... here's where things get a little hairy, because I want to do some slight history rewriting so that everything is copacetic in the new repo. So, I decide to change my approach, and go with what turned out to be a simpler strategy.

Over in our new react-circle-of-fifths repository, I make a quick initial commit with a README, which gives us a point to attach the rest of our history to. Then, using the patch-based strategy from the StackOverflow thread, I generate a patch for only the two files I care about. This seemed a little easier than trying to come up with the right filter-branch invocation.

~/src/practicker-tools-extract master
❯ git log
    --pretty=email
    --patch-with-stat
    --reverse
    --full-index
    --binary
    -m
    --first-parent
    -- src/CircleOfFifths.* > patch
    

This results in a patch file, which I can easily import into my new project using git am. I cut a new branch in that repo, called devblog, and import the commits:

~/src/react-circle-of-fifths devblog
❯ git am --committer-date-is-author-date < ../practicker-tools-extract/patch
    

Now, I've got the files for my component, along with their full commit history, in the right spot in the new repo. This means we're ready to start working on turning the Circle of Fifths into a standalone component, which we can publish to NPM and make available to the world!

Preparing Circle Of Fifth's New Repo

Now, we'll drop back to our main branch. We'll add a couple important supporting files to the repository, which we need to make this a standalone project. Then, we'll rebase our newly created devblog branch onto main, making all the history look like it was written by someone who started out with a plan.

First, we run yarn init to set up our package.json file, which provides a bunch of important information about our package. Since I want this to be public, I took the time to go search the NPM registry, and make sure the name I wanted was available. I also made an empty public repo on my GitHub account to host the source code. With that out of the way, initializing the package just takes some quick copy-paste action:

~/src/react-circle-of-fifths main*
❯ yarn init
yarn init v1.22.19
question name (react-circle-of-fifths):
question version (1.0.0): 0.0.1
question description: An interactive Circle of Fifths component for React
question entry point (index.js):
question repository url: https://github.com/epiccoleman/react-circle-of-fifths
question author (Eric Coleman ):
question license (MIT):
question private:
success Saved package.json
✨  Done in 68.50s.
    

I also added a copy of the MIT License to the repository. This license is about as permissive as it gets, and grants anyone permission to use the code for any purpose they like.

Now, since this component originally got all its dependencies from the practicker-tools project, we'll need to add those. Right now, we're using react, react-dom, and typescript. Typescript will only be needed as a development dependency - when we're ready to publish, we'll transpile our Typescript code to Javascript with tsc, so that projects that use our component can decide for themselves whether to use Typescript or not. As for react and react-dom, we'll declare these as peerDependencies. This means that we aren't going to ship the whole React library with our component - rather, we'll expect that users of the component already have React installed in their project, and utilize their local copy of the React.

We add the dependencies like so - first typescript as a dev dependency, and react and react-dom as peer dependencies.

yarn add --dev typescript
yarn add --peer react react-dom
    

Here is the package.json file at this point in the process.

At this point, our project represents a complete React component. Now, we just need to set up a process for publishing the component.

Preparing react-circle-of-fifths for Publication

The first thing we need to do is set up a transpilation process, for converting the Typescript code of our component into plain Javascript. We can configure tsc - the Typescript compiler - with a tsconfig.json file. After fiddling around a bit, and reading up on the various options, this is what I came up with.

I've added some comments to this code block to explain what each option does. Note that while comments are not usually syntactically valid in JSON, they are allowed in tsconfig.json, so these comments will compile!

Now, let's run yarn tsc to see what happens. This results in two files in our lib directory - the generated JS source of the component, and a source map file.

├── lib
│   ├── CircleOfFifths.js
│   └── CircleOfFifths.js.map
    

I also notice that we're currently not adding the component's CSS file to our bundle - so we'll just copy that in manually. Instead of remembering to do that every time, we'll just add a build task to our package.json file. This cleans up any previously compiled code, runs tsc, and then copies our component's CSS file into the lib directory.

Next, we'll need a sensible entry point for our component. We'll add an index.ts file to our source code, which simply exports the component. We also set the entryPoint to the generated index.js file in our package.json file.

Now, we're ready to try using our component. The source is compiled, and everything seems to be working as expected. First, we'll just try a local import. Providing npm (or yarn) with a filesystem path will behave just like a package downloaded from the NPM registry.

I make a copy of my React template project, and install the package directly from my filesystem, like this:

cp -r parcel-react-tailwind-template test-circle-of-fifths
cd test-circle-of-fifths
yarn add ../react-circle-of-fifths
    

Then, all I have to do is import the component in my template's App component, and render it:

And look at that - it works!

Publishing react-circle-of-fifths to NPM

At this point, we've proven that this repository is ready to be published, and pulled in by anyone who wants to use it. There are plenty of tweaks and features still needed at this point, and it might be a little early to really "release." But I'm getting excited, so let's try pushing it up with an "alpha" version number, and we'll try pulling it down into our test project.

I make an account over on npmjs.com, and follow the setup instructions from the page on Publishing Unscoped Packages. This is really pretty slick - all I have to do is run npm adduser to do the initial setup of my account, and then run npm publish in the root directory of the package. Just like that, I have my very own page on npm.js for react-circle-of-fifths. Groovy.

Back in my test project, I remove the local dependency on my package:

yarn remove react-circle-of-fifths
    

And then, re-add it - but instead of using a local path to the package, I just refer to it by name:

yarn add react-circle-of-fifths
    

The package is downloaded from the registry, installed into my test project, and renders just as successfully as before. It's official - this project is live, published, and ready for use!

Integrating Our New Package

The last thing I want to do is to delete the source code for the CircleOfFifths component from the key-thing repo, and install my dependency there. In this case, I used a yarn link command to add the dependency - because I'm going to need to work on both the app, and mess with the component at the same time. This completes the "divorce" between the app I'm building and the component itself.

Everything works great - except for one little detail.

My initial suspicion was that, because I had removed the surrounding <div> element from the extracted component, I'd broken some of my app's styling. However, after further examination, it turns out that I didn't provide a good way to style the <<svg> element that the component renders.

Although this is annoying, this is a perfect example of why I went to the effort to pull out this component. Now that I have to use the component through its public API, I'll be able to catch issues like this, and ultimately, design it better!

Closing Thoughts

Wow, once again, that turned out to be quite a long post. This has been a really educational side project, because it's a good demonstration of one of the hard truths of software engineering - much of the time, just writing the code is the easy part.

When we decide to go the extra mile, and write code in a way that it can be consumed by many users, there is a lot of extra work that needs to be done! But the benefit of this work is that it forces us to think harder about the things we build and how they will be used, and I think that this process, while sometimes grueling, results in a lot better software at the end of it all. In my next posts, we'll cover some of the things I've done to make this component ready for integration into other projects - like enabling selection management, custom styling, and adding a suite of tests.

If this component is something you're interested in using, you'll be happy to read that I just recently published the 1.0.0 version, which has a few cool features that this series of posts hasn't covered yet. I've got a lot of big ideas for this project, but I think the 1.0.0 release represents the first time I've really felt like it's ready for public consumption. Learn more at the react-circle-of-fifths GitHub Page or its page on npmjs.com

As always, thanks for reading, and see you next time!

[1] If you're all the way down here, you may realize that this sentence was very overly optimistic. 😅