Git Magic

I’ve played computer games almost all my life. In contrast, I only started using version control systems as an adult. I suspect I’m not alone, and comparing the two may make these concepts easier to explain and understand.

Think of editing your code or document, or whatever, as playing a game. Once you’ve made a lot of progress, you’d like to save. To do so, you click on the "Save" button in your trusty editor.

But this will overwrite the old version. It’s like those old school games which only had one save slot: sure you could save, but you could never go back to an older state. Which was a shame, because your previous save might have been right at an exceptionally fun part of the game that you’d like to revisit one day. Or worse still, your current save is in an unwinnable state, and you have to start again.

When editing, you can "Save As…" a different file, or copy the file somewhere first before saving if you want to savour old versions. You can compress them too to save space. This is a primitive and labour-intensive form of version control. Computer games improved on this long ago, many of them providing multiple automatically timestamped save slots.

Let’s make the problem slightly tougher. Say you have a bunch of files that go together, such as source code for a project, or files for a website. Now if you want to keep an old version you have to archive a whole directory. Keeping many versions around by hand is inconvenient, and quickly becomes expensive.

With some computer games, a saved game really does consist of a directory full of files. These games hide this detail from the player and present a convenient interface to manage different versions of this directory.

Version control systems are no different. They all have nice interfaces to manage a directory of stuff. You can save the state of the directory every so often, and you can load any one of the saved states later on. Unlike most computer games, they’re usually smart about conserving space. Typically, only a few files change between version to version, and not by much. Storing the differences instead of entire new copies saves room.

Now imagine a very difficult computer game. So difficult to finish that many experienced gamers all over the world decide to team up and share their saved games to try to beat it. Speedruns are real-life examples: players specializing in different levels of the same game collaborate to produce amazing results.

How would you set up a system so they can get at each other’s saves easily? And upload new ones?

In the old days, every project used centralized version control. A server somewhere held all the saved games. Nobody else did. Every player kept at most a few saved games on their machine. When a player wanted to make progress, they’d download the latest save from the main server, play a while, save and upload back to the server for everyone else to use.

What if a player wanted to get an older saved game for some reason? Maybe the current saved game is in an unwinnable state because somebody forgot to pick up an object back in level three, and they want to find the latest saved game where the game can still be completed. Or maybe they want to compare two older saved games to see how much work a particular player did.

There could be many reasons to want to see an older revision, but the outcome is the same. They have to ask the central server for that old saved game. The more saved games they want, the more they need to communicate.

The new generation of version control systems, of which Git is a member, are known as distributed systems, and can be thought of as a generalization of centralized systems. When players download from the main server they get every saved game, not just the latest one. It’s as if they’re mirroring the central server.

This initial cloning operation can be expensive, especially if there’s a long history, but it pays off in the long run. One immediate benefit is that when an old save is desired for any reason, communication with the central server is unnecessary.

For this topic, our computer game analogy becomes too thinly stretched. Instead, let us again consider editing a document.

Suppose Alice inserts a line at the beginning of a file, and Bob appends one at the end of his copy. They both upload their changes. Most systems will automatically deduce a reasonable course of action: accept and merge their changes, so both Alice’s and Bob’s edits are applied.

Now suppose both Alice and Bob have made distinct edits to the same line. Then it is impossible to proceed without human intervention. The second person to upload is informed of a merge conflict, and must choose one edit over another, or revise the line entirely.

More complex situations can arise. Version control systems handle the simpler cases themselves, and leave the difficult cases for humans. Usually their behaviour is configurable.

About to attempt something drastic? Before you do, take a snapshot of all files in the current directory with:

$ git init
$ git add .
$ git commit -m "My first backup"

Now if your new edits go awry, restore the pristine version:

$ git reset --hard

To save the state again:

$ git commit -a -m "Another backup"

$ git add NEWFILES...

$ git rm OLDFILES...

$ git mv OLDFILE NEWFILE

Sometimes you just want to go back and forget about every change past a certain point because they’re all wrong. Then:

$ git log

shows you a list of recent commits, and their SHA1 hashes. Next, type:

$ git reset --hard SHA1_HASH

to restore the state to a given commit and erase all newer commits from the record permanently.

Other times you want to hop to an old state briefly. In this case, type:

$ git checkout SHA1_HASH

This takes you back in time, while preserving newer commits. However, like time travel in a science-fiction movie, if you now edit and commit, you will be in an alternate reality, because your actions are different to what they were the first time around.

This alternate reality is called a branch, and we’ll have more to say about this later. For now, just remember that

$ git checkout master

will take you back to the present. Also, to stop Git complaining, always commit or reset your changes before running checkout.

To take the computer game analogy again:

You can choose only to restore particular files and subdirectories by appending them after the command:

$ git checkout SHA1_HASH some.file another.file

Take care, as this form of checkout can silently overwrite files. To prevent accidents, commit before running any checkout command, especially when first learning Git. In general, whenever you feel unsure about any operation, Git command or not, first run git commit -a.

Don’t like cutting and pasting hashes? Then use:

$ git checkout :/"My first b"

to jump to the commit that starts with a given message. You can also ask for the 5th-last saved state:

$ git checkout master~5

$ git commit -a
$ git revert SHA1_HASH

Some projects require a changelog. Generate one by typing:

$ git log > ChangeLog

Get a copy of a project managed with Git by typing:

$ git clone git://server/path/to/files

For example, to get all the files I used to create this site:

$ git clone git://git.or.cz/gitmagic.git

We’ll have much to say about the clone command soon.

If you’ve already downloaded a copy of a project using git clone, you can upgrade to the latest version with:

$ git pull

Suppose you’ve written a script you’d like to share with others. You could just tell them to download from your computer, but if they do so while you’re improving the script or making experimental changes, they could wind up in trouble. Of course, this is why release cycles exist. Developers may work on a project frequently, but they only make the code available when they feel it is presentable.

To do this with Git, in the directory where your script resides:

$ git init
$ git add .
$ git commit -m "First release"

Then tell your users to run:

$ git clone your.computer:/path/to/script

to download your script. This assumes they have ssh access. If not, run git daemon and tell your users to instead run:

$ git clone git://your.computer/path/to/script

From now on, every time your script is ready for release, execute:

$ git commit -a -m "Next release"

and your users can upgrade their version by changing to the directory containing your script and typing:

$ git pull

Your users will never end up with a version of your script you don’t want them to see. Obviously this trick works for anything, not just scripts.

Find out what changes you’ve made since the last commit with:

$ git diff

Or since yesterday:

$ git diff "@{yesterday}"

Or between a particular version and 2 versions ago:

$ git diff SHA1_HASH "master~2"

In each case the output is a patch that can be applied with git apply. Try also:

$ git whatchanged --since="2 weeks ago"

Often I’ll browse history with qgit instead, due to its slick photogenic interface, or tig, a text-mode interface that works well over slow connections. Alternatively, install a web server, run git instaweb and fire up any web browser.

Let A, B, C, D be four successive commits where B is the same as A except some files have been removed. We want to add the files back at D and not at B. How can we do this?

There are at least three solutions. Assuming we are at D:

Which choice is best? Whichever you prefer most. It is easy to get what you want with Git, and often there are many ways to get it.

This is the reason I first used Git. I can tolerate making tarballs or using rsync for backups and basic syncing. But sometimes I edit on my laptop, other times on my desktop, and the two may not have talked to each other in between.

Initialize a Git repository and commit your files on one machine. Then on the other:

$ git clone other.computer:/path/to/files

to create a second copy of the files and Git repository. From now on,

$ git commit -a
$ git pull other.computer:/path/to/files HEAD

will pull in the state of the files on the other computer into the one you’re working on. If you’ve recently made conflicting edits in the same file, Git will let you know and you should commit again after resolving them.

Initialize a Git repository for your files:

$ git init
$ git add .
$ git commit -m "Initial commit"

On the central server, initialize an empty Git repository with some name, and start the Git daemon if necessary:

$ GIT_DIR=proj.git git init
$ git daemon --detach  # it might already be running

For Git hosting services, follow the instructions to setup the initially empty Git repository. Typically one fills in a form on a webpage.

Push your project to the central server with:

$ git push git://central.server/path/to/proj.git HEAD

We’re ready. To check out source, a developer types

$ git clone git://central.server/path/to/proj.git

After making changes, the code is checked in to the main server by:

$ git commit -a
$ git push

If the main server has been updated, the latest version needs to be checked out before the push. To sync to the latest version:

$ git commit -a
$ git pull

Sick of the way a project is being run? Think you could do a better job? Then on your server:

$ git clone git://main.server/path/to/files

Next tell everyone about your fork of the project at your server.

At any later time, you can merge in the changes from the original project with:

$ git pull

Want numerous tamper-proof geographically diverse redundant archives? If your project has many developers, don’t do anything! Every clone of your code is effectively a backup. Not just of the current state, but of your project’s entire history. Thanks to cryptographic hashing, if anyone’s clone becomes corrupted, it will be spotted as soon as they try to communicate with others.

If your project is not so popular, find as many servers as you can to host clones.

The truly paranoid should always write down the latest 20-byte SHA1 hash of the HEAD somewhere safe. It has to be safe, not private. For example, publishing it in a newspaper would work well, because it’s hard for an attacker to alter every copy of a newspaper.

Say you want to work on several features in parallel. Then commit your project and run:

$ git clone . /some/new/directory

Git exploits hard links and file sharing as much as safely possible to create this clone, so it will be ready in a flash, and you can now work on two independent features simultaneously. For example, you can edit one clone while the other is compiling.

At any time, you can commit and pull changes from the other clone.

$ git pull /the/other/clone HEAD

Are you working on a project that uses some other version control system, and you sorely miss Git? Then initialize a Git repository in your working directory:

$ git init
$ git add .
$ git commit -m "Initial commit"

then clone it:

$ git clone . /some/new/directory

Now go to the new directory and work here instead, using Git to your heart’s content. Once in a while, you’ll want to sync with everyone else, in which case go to the original directory, sync using the other version control system, and type:

$ git add .
$ git commit -m "Sync with everyone else"

Then go to the new directory and run:

$ git commit -a -m "Description of my changes"
$ git pull

The procedure for giving your changes to everyone else depends on the other version control system. The new directory contains the files with your changes. Run whatever commands of the other version control system are needed to upload them to the central repository.

The git svn command automates the above for Subversion repositories, and can also be used to export a Git project to a Subversion repository.

Ever play one of those games where at the push of a button ("the boss key"), the screen would instantly display a spreadsheet or something? So if the boss walked in the office while you were playing the game you could quickly hide it away?

In some directory:

$ echo "I'm smarter than my boss" > myfile.txt
$ git init
$ git add .
$ git commit -m "Initial commit"

We have created a Git repository that tracks one text file containing a certain message. Now type:

$ git checkout -b boss  # nothing seems to change after this
$ echo "My boss is smarter than me" > myfile.txt
$ git commit -a -m "Another commit"

It looks like we’ve just overwritten our file and committed it. But it’s an illusion. Type:

$ git checkout master  # switch to original version of the file

and hey presto! The text file is restored. And if the boss decides to snoop around this directory, type:

$ git checkout boss  # switch to version suitable for boss' eyes

You can switch between the two versions of the file as much as you like, and commit to each independently.

Say you’re working on some feature, and for some reason, you need to go back to an old version and temporarily put in a few prints statements to see how something works. Then:

$ git commit -a
$ git checkout SHA1_HASH

Now you can add ugly temporary code all over the place. You can even commit these changes. When you’re done,

$ git checkout master

to return to your original work. Observe that any uncommitted changes are carried over.

What if you wanted to save the temporary changes after all? Easy:

$ git checkout -b dirty

and commit before switching back to the master branch. Whenever you want to return to the dirty changes, simply type

$ git checkout dirty

We touched upon this command in an earlier chapter, when discussing loading old states. At last we can tell the whole story: the files change to the requested state, but we must leave the master branch. Any commits made from now on take your files down a different road, which can be named later.

In other words, after checking out an old state, Git automatically puts you in a new, unnamed branch, which can be named and saved with git checkout -b.

You’re in the middle of something when you are told to drop everything and fix a newly discovered bug:

$ git commit -a
$ git checkout -b fixes SHA1_HASH

Then once you’ve fixed the bug:

$ git commit -a -m "Bug fixed"
$ git push  # to the central repository
$ git checkout master

and resume work on your original task.

Often in hardware projects, the second step of a plan must wait for the first step to be completed before it can begin. A car undergoing repairs might sit idly in a garage until a particular part arrives from the factory. A prototype might wait for a chip to be fabricated before construction can continue.

Software projects can be similar. The second part of a new feature may have to wait until the first part has been released and tested. Some projects require your code to be reviewed before accepting it, so you might wait until the first part is approved before starting the second part.

In Git, thanks to painless branching and merging, we can bend the rules and work on Part II before Part I is officially ready. Suppose you have committed Part I and sent it for review. Let’s say you’re in the master branch. Then branch off:

$ git checkout -b part2

Next, work on Part II, committing your changes along the way. To err is human, and often you’ll want to go back and fix something in Part I. If you’re lucky, or very good, you can skip these lines.

$ git checkout master  # Go back to Part I.
$ edit files           # Fix Part I.
$ git checkout part2   # Go back to Part II.
$ git merge master     # Merge in those fixes.

Eventually, Part I is approved:

$ git checkout master  # Go back to Part I.
$ some_command         # Some command you're supposed to run when the
                       # current working directory is officially ready.
$ git merge part2      # Merge in Part II.
$ git branch -d part2

Now you’re in the master branch again, with Part II in the working directory.

It’s easy to extend this trick for any number of parts. It’s also easy to branch off retroactively: suppose you belatedly realize you should have created a branch several commits ago. Then type:

$ git branch -m master part2   # Rename "master" branch to "part2".
$ git checkout SHA1 -b master  # The commit representing Part I.

The master branch now contains just Part I, and the part2 branch contains the rest.

Perhaps you like to work on all aspects of a project in the same branch. You want to keep works-in-progress to yourself and want others to see your commits only when they have been neatly organized. Start a couple of branches:

$ git checkout -b sanitized
$ git checkout -b medley

Next, work on anything: fix bugs, add features, add temporary code, and so forth, committing often along the way. Then:

$ git checkout sanitized
$ git cherry-pick SHA1_HASH

applies a given commit to the "sanitized" branch. With appropriate cherry-picks you can construct a branch that contains only permanent code, and has related commits grouped together.

List all branches by typing:

$ git branch

By default, you start in a branch named "master". Some advocate leaving the "master" branch untouched and creating new branches for your own edits.

The -d and -m options allow you to delete and move (rename) branches. See git help branch.

The "master" branch is a useful convention. Others may assume that your repository has a branch with this name, and that it contains the official version of your project. You can rename or obliterate the "master" branch, but you might as well respect this custom.

After a while you may realize you are creating short-lived branches frequently for similar reasons: every other branch merely serves to save the current state so you can briefly hop back to an older state to fix a high-priority bug or something.

It’s analogous to changing the TV channel temporarily to see what else is on. But instead of pushing a couple of buttons, you have to create, check out and delete temporary branches and commits. Luckily, Git has a shortcut that is as convenient as a TV remote control:

$ git stash

This saves the current state in a temporary location (a stash) and restores the previous state. Your working directory appears exactly as it was before you started editing, and you can fix bugs, pull in upstream changes, and so on. When you want to go back to the stashed state, type:

$ git stash apply  # You may need to resolve some conflicts.

You can have multiple stashes, and manipulate them in various ways. See git help stash. As you may have guessed, Git maintains branches behind the scenes to perform this magic trick.

Applications such as Mozilla Firefox allow you to open multiple tabs and multiple windows. Switching tabs gives you different content in the same window. Git branching is like tabs for your working directory. Continuing this analogy, Git cloning is like opening a new window. Being able to do both improves the user experience.

On a higher level, several window managers support multiple desktops. Branching in Git is similar to switching to a different desktop, while cloning is similar to attaching another monitor to gain another desktop.

Yet another example is the screen utility. This gem lets you create, destroy and switch between multiple terminal sessions in the same terminal. Instead of opening new terminals (clone), you can use the same one if you run screen (branch). In fact, you can do a lot more with screen but that’s a topic for another text.

Cloning, branching, and merging are fast and local in Git, encouraging you to use the combination that best suits you. Git lets you work exactly how you want.

Did you just commit, but wish you had typed a different message? Then run:

$ git commit --amend

to change the last message. Realized you forgot to add a file? Run git add to add it, and then run the above command.

Want to include a few more edits in that last commit? Then make those edits and run:

$ git commit --amend -a

Let’s suppose the previous problem is ten times worse. After a lengthy session you’ve made a bunch of commits. But you’re not quite happy with the way they’re organized, and some of those commit messages could use rewording. Then type:

$ git rebase -i HEAD~10

and the last 10 commits will appear in your favourite $EDITOR. A sample excerpt:

pick 5c6eb73 Added repo.or.cz link
pick a311a64 Reordered analogies in "Work How You Want"
pick 100834f Added push target to Makefile

Then:

If you marked a commit for editing, then run:

$ git commit --amend

Otherwise, run:

$ git rebase --continue

So commit early and commit often: you can easily tidy up later with rebase.

You’re working on an active project. You make some local commits over time, and then you sync with the official tree with a merge. This cycle repeats itself a few times before you’re ready to push to the central tree.

But now the history in your local Git clone is a messy jumble of your changes and the official changes. You’d prefer to see all your changes in one contiguous section, and after all the official changes.

This is a job for git rebase as described above. In many cases you can use the --onto flag and avoid interaction.

Also see git help rebase for detailed examples of this amazing command. You can split commits. You can even rearrange branches of a tree.

Occasionally, you need the source control equivalent of airbrushing people out of official photos, erasing them from history in a Stalinesque fashion. For example, suppose we intend to release a project, but it involves a file that should be kept private for some reason. Perhaps I left my credit card number in a text file and accidentally added it to the project. Deleting the file is insufficient, for the file can be accessed from older commits. We must remove the file from all commits:

$ git filter-branch --tree-filter 'rm top/secret/file' HEAD

See git help filter-branch, which discusses this example and gives a faster method. In general, filter-branch lets you alter large sections of history with a single command.

Afterwards, the .git/refs/original directory describes the state of affairs before the operation. Check the filter-branch command did what you wanted, then delete this directory if you wish to run more filter-branch commands.

Lastly, replace clones of your project with your revised version if you want to interact with them later.

Want to migrate a project to Git? If it’s managed with one of the more well-known systems, then chances are someone has already written a script to export the whole history to Git.

Otherwise, look up git fast-import, which reads text input in a specific format to create Git history from scratch. Typically a script using this command is hastily cobbled together and run once, migrating the project in a single shot.

As an example, paste the following listing into temporary file, such as /tmp/history:

commit refs/heads/master
committer Alice <alice@example.com> Thu, 01 Jan 1970 00:00:00 +0000
data <<EOT
Initial commit.
EOT

M 100644 inline hello.c
data <<EOT
#include <stdio.h>

int main() {
  printf("Hello, world!\n");
  return 0;
}
EOT


commit refs/heads/master
committer Bob <bob@example.com> Tue, 14 Mar 2000 01:59:26 -0800
data <<EOT
Replace printf() with write().
EOT

M 100644 inline hello.c
data <<EOT
#include <unistd.h>

int main() {
  write(1, "Hello, world!\n", 14);
  return 0;
}
EOT

Then create a Git repository from this temporary file by typing:

$ mkdir project; cd project; git init
$ git fast-import < /tmp/history

You can checkout the latest version of the project with:

$ git checkout master .

The git fast-export command converts any git repository to the git fast-import format, whose output you can study for writing exporters, and also to transport git repositories in a human-readable format. Indeed, these commands can send repositories of text files over text-only channels.

You’ve just discovered a broken feature in your program which you know for sure was working a few months ago. Argh! Where did this bug come from? If only you had been testing the feature as you developed.

It’s too late for that now. However, provided you’ve been committing often, Git can pinpoint the problem:

$ git bisect start
$ git bisect bad SHA1_OF_BAD_VERSION
$ git bisect good SHA1_OF_GOOD_VERSION

Git checks out a state halfway in between. Test the feature, and if it’s still broken:

$ git bisect bad

If not, replace "bad" with "good". Git again transports you to a state halfway between the known good and bad versions, narrowing down the possibilities. After a few iterations, this binary search will lead you to the commit that caused the trouble. Once you’ve finished your investigation, return to your original state by typing:

$ git bisect reset

Instead of testing every change by hand, automate the search by running:

$ git bisect run COMMAND

Git uses the return value of the given command, typically a one-off script, to decide whether a change is good or bad: the command should exit with code 0 when good, 125 when the change should be skipped, and anything else between 1 and 127 if it is bad. A negative return value aborts the bisect.

You can do much more: the help page explains how to visualize bisects, examine or replay the bisect log, and eliminate known innocent changes for a speedier search.

Like many other version control systems, Git has a blame command:

$ git blame FILE

which annotates every line in the given file showing who last changed it, and when. Unlike many other version control systems, this operation works offline, reading only from local disk.

In a centralized version control system, history modification is a difficult operation, and only available to administrators. Cloning, branching, and merging are impossible without network communication. So are basic operations such as browsing history, or committing a change. In some systems, users require network connectivity just to view their own changes or open a file for editing.

Centralized systems preclude working offline, and need more expensive network infrastructure, especially as the number of developers grows. Most importantly, all operations are slower to some degree, usually to the point where users shun advanced commands unless absolutely necessary. In extreme cases this is true of even the most basic commands. When users must run slow commands, productivity suffers because of an interrupted work flow.

I experienced these phenomena first-hand. Git was the first version control system I used. I quickly grew accustomed to it, taking many features for granted. I simply assumed other systems were similar: choosing a version control system ought to be no different from choosing a text editor or web browser.

I was shocked when later forced to use a centralized system. A flaky internet connection matters little with Git, but makes development unbearable when it needs to be as reliable as local disk. Additionally, I found myself conditioned to avoid certain commands because of the latencies involved, which ultimately prevented me from following my desired work flow.

When I had to run a slow command, the interruption to my train of thought dealt a disproportionate amount of damage. While waiting for server communication to complete, I’d do something else to pass the time, such as check email or write documentation. By the time I returned to the original task, the command had finished long ago, and I would waste more time trying to remember what I was doing. Humans are bad at context switching.

There was also an interesting tragedy-of-the-commons effect: anticipating network congestion, individuals would consume more bandwidth than necessary on various operations in an attempt to reduce future delays. The combined efforts intensified congestion, encouraging individuals to consume even more bandwidth next time to avoid even longer delays.

Every commit has an author name and email, which is shown by git log. By default, Git uses system settings to populate these fields. To set them explicitly, type:

$ git config --global user.name "John Doe"
$ git config --global user.email johndoe@example.com

Omit the global flag to set these options only for the current repository.

Suppose you have SSH access to a web server, but Git is not installed. Though less efficient than its native protocol, Git can communicate over HTTP.

Download, compile and install Git in your account, and create a repository in your web directory:

$ GIT_DIR=proj.git git init

In the "proj.git" directory, run:

$ git --bare update-server-info
$ cp hooks/post-update.sample hooks/post-update

For older versions of Git, the copy command fails and you should run:

$ chmod a+x hooks/post-update

Now you can publish your latest edits via SSH from any clone:

$ git push web.server:/path/to/proj.git master

and anybody can get your project with:

$ git clone http://web.server/proj.git

Want to synchronize repositories without servers, or even a network connection? Need to improvise during an emergency? We’ve seen git fast-export and git fast-import can convert repositories to a single file and back. We could shuttle such files back and forth to transport git repositories over any medium, but a more efficient tool is git bundle.

The sender creates a bundle:

$ git bundle create somefile HEAD

then transports the bundle, somefile, to the other party somehow: email, thumb drive, floppy disk, an xxd printout and an OCR machine, reading bits over the phone, smoke signals, etc. The receiver retrieves commits from the bundle by typing:

$ git pull somefile

The receiver can even do this from an empty repository. Despite its size, somefile contains the entire original git repository.

In larger projects, eliminate waste by bundling only changes the other repository lacks:

$ git bundle create somefile HEAD ^COMMON_SHA1

If done frequently, one could easily forget which commit was last sent. The help page suggests using tags to solve this. Namely, after you send a bundle, type:

$ git tag -f lastbundle HEAD

and create new refresher bundles with:

$ git bundle create newbundle HEAD ^lastbundle

Patches are text representations of your changes that can be easily understood by computers and humans alike. This gives them universal appeal. You can email a patch to developers no matter what version control system they’re using. As long as your audience can read their email, they can see your edits. Similarly, on your side, all you require is an email account: there’s no need to setup an online Git repository.

Recall from the first chapter:

$ git diff COMMIT

outputs a patch which can be pasted into an email for discussion. In a Git repository, type:

$ git apply < FILE

to apply the patch.

In more formal settings, when author names and perhaps signatures should be recorded, generate the corresponding patches past a certain point by typing:

$ git format-patch START_COMMIT

The resulting files can be given to git-send-email, or sent by hand. You can also specify a range of commits:

$ git format-patch START_COMMIT..END_COMMIT

On the receiving end, save an email to a file, then type:

$ git am < FILE

This applies the incoming patch and also creates a commit, including information such as the author.

With a browser email client, you may need to click a button to see the email in its raw original form before saving the patch to a file.

There are slight differences for mbox-based email clients, but if you use one of these, you’re probably the sort of person who can figure them out easily without reading tutorials!

After cloning a repository, running git push or git pull will automatically push to or pull from the original URL. How does Git do this? The secret lies in config options initialized created with the clone. Let’s take a peek:

$ git config --list

The remote.origin.url option controls the source URL; "origin" is a nickname given to the source repository. As with the "master" branch convention, we may change or delete this nickname but there is usually no reason for doing so.

If the original repository moves, we can update the URL via:

$ git config remote.origin.url NEW_URL

The branch.master.merge option specifies the default remote branch in a git pull. During the initial clone, it is set to the current branch of the source repository, so even if the HEAD of the source repository subsequently moves to a different branch, a later pull will faithfully follow the original branch.

This option only applies to the repository we first cloned from, which is recorded in the option branch.master.remote. If we pull in from other repositories we must explicitly state which branch we want:

$ git pull ANOTHER_URL master

The above explains why some of our earlier push and pull examples had no arguments.

When you clone a repository, you also clone all its branches. You may not have noticed this because Git hides them away: you must ask for them specifically. This prevents branches in the remote repository from interfering with your branches, and also makes Git easier for beginners.

List the remote branches with:

$ git branch -r

You should see something like:

origin/HEAD
origin/master
origin/experimental

These represent branches and the HEAD of the remote repository, and can be used in regular Git commands. For example, suppose you have made many commits, and wish to compare against the last fetched version. You could search through the logs for the appropriate SHA1 hash, but it’s much easier to type:

$ git diff origin/HEAD

Or you can see what the "experimental" branch has been up to:

$ git log origin/experimental

Suppose two other developers are working on our project, and we want to keep tabs on both. We can follow more than one repository at a time with:

$ git remote add other ANOTHER_URL
$ git pull other some_branch

Now we have merged in a branch from the second repository, and we have easy access to all branches of all repositories:

$ git diff origin/experimental^ other/some_branch~5

But what if we just want to compare their changes without affecting our own work? In other words, we want to examine their branches without having their changes invade our working directory. In this case, rather than pull, run:

$ git fetch        # Fetch from origin, the default.
$ git fetch other  # Fetch from the second programmer.

This fetches their histories and nothing more, so although the working directory remains untouched, we can refer to any branch of any repository in a Git command. By the way, behind the scenes, a pull is simply a fetch followed by git merge; recall the latter merges a given commit into the working directory. Usually we pull because we want to merge after a fetch; this situation is a notable exception.

See git help remote for how to remove remote repositories, ignore certain branches, and more.

For my projects, I like contributors to prepare Git repositories which I can pull. Some Git hosting services let you host your own fork of a project with the click of a button.

After I fetch a tree, I run Git commands to navigate and examine the changes, which ideally are well-organized and well-described. I merge unpublished changes of my own, and perhaps make further edits. Once satisfied, I push to the official repository.

Though I infrequently receive contributions, I believe this approach scales well. See this blog post by Linus Torvalds.

Staying in the Git world is slightly more convenient than patch files, as it saves me the step of converting them to Git commits. Furthermore, Git automatically handles details such as recording the author’s name and email address, as well as the time and date, and asks the author to describe their own change.

For my projects, Git tracks exactly the files I’d like to archive and release to users. To create a tarball of the source code, I run:

$ git archive --format=tar --prefix=proj-1.2.3/ HEAD

Telling Git when you’ve added, deleted and renamed files is troublesome for certain projects. Instead, you can type:

$ git add .
$ git add -u

Git will look at the files in the current directory and work out the details by itself. Instead of the second add command, run git commit -a if you also intend to commit at this time. See git help ignore for how to specify files that should be ignored.

You can perform the above in a single pass with:

$ git ls-files -d -m -o -z | xargs -0 git update-index --add --remove

The -z and -0 options prevent ill side-effects from filenames containing strange characters. As this command adds ignored files, you may want to use the -x or -X option.

Have you neglected to commit for too long? Been coding furiously and forgotten about source control until now? Made a series of unrelated changes, because that’s your style?

No worries. Run:

$ git add -p

For each edit you made, Git will show you the hunk of code that was changed, and ask if it should be part of the next commit. Answer with "y" or "n". You have other options, such as postponing the decision; type "?" to learn more.

Once you’re satisfied, type

$ git commit

to commit precisely the changes you selected (the staged changes). Make sure you omit the -a option, otherwise Git will commit all the edits.

What if you’ve edited many files in many places? Reviewing each change one by one becomes frustratingly mind-numbing. In this case, use git add -i, whose interface is less straightforward, but more flexible. With a few keystrokes, you can stage or unstage several files at a time, or review and select changes in particular files only. Alternatively, run git commit --interactive which automatically commits after you’re done.

The HEAD tag is like a cursor that normally points at the latest commit, advancing with each new commit. Some Git commands let you move it. For example:

$ git reset HEAD~3

will move the HEAD three commits back. Thus all Git commands now act as if you hadn’t made those last three commits, while your files remain in the present. See the help page for some applications.

But how can you go back to the future? The past commits know nothing of the future.

If you have the SHA1 of the original HEAD then:

$ git reset SHA1

But suppose you never took it down? Don’t worry, for commands like these, Git saves the original HEAD as a tag called ORIG_HEAD, and you can return safe and sound with:

$ git reset ORIG_HEAD

Perhaps ORIG_HEAD isn’t enough. Perhaps you’ve just realized you made a monumental mistake and you need to go back to an ancient commit in a long-forgotten branch.

By default, Git keeps a commit for at least two weeks, even if you ordered Git to destroy the branch containing it. The trouble is finding the appropriate hash. You could look at all the hash values in .git/objects and use trial and error to find the one you want. But there’s a much easier way.

Git records every hash of a commit it computes in .git/logs. The subdirectory refs contains the history of all activity on all branches, while the file HEAD shows every hash value it has ever taken. The latter can be used to find hashes of commits on branches that have been accidentally lopped off.

The reflog command provides a friendly interface to these log files. Try

$ git reflog

Instead of cutting and pasting hashes from the reflog, try:

$ git checkout "@{10 minutes ago}"

Or checkout the 5th-last visited commit via:

$ git checkout "@{5}"

See the “Specifying Revisions” section of git help rev-parse for more.

You may wish to configure a longer grace period for doomed commits. For example:

$ git config gc.pruneexpire "30 days"

means a deleted commit will only be permanently lost once 30 days have passed and git gc is run.

You may also wish to disable automatic invocations of git gc:

$ git config gc.auto 0

in which case commits will only be deleted when you run git gc manually.

In true UNIX fashion, Git’s design allows it to be easily used as a low-level component of other programs, such as GUI and web interfaces, alternative command-line interfaces, patch managements tools, importing and conversion tools and so on. In fact, some Git commands are themselves scripts standing on the shoulders of giants. With a little tinkering, you can customize Git to suit your preferences.

One easy trick is to use built-in Git aliases to shorten your most frequently used commands:

$ git config --global alias.co checkout
$ git config --global --get-regexp alias  # display current aliases
alias.co checkout
$ git co foo                              # same as 'git checkout foo'

Another is to print the current branch in the prompt, or window title. Invoking

$ git symbolic-ref HEAD

shows the current branch name. In practice, you most likely want to remove the "refs/heads/" and ignore errors:

$ git symbolic-ref HEAD 2> /dev/null | cut -b 12-

The contrib subdirectory is a treasure trove of tools built on Git. In time, some of them may be promoted to official commands. On Debian and Ubuntu, this directory lives at /usr/share/doc/git-core/contrib.

One popular resident is workdir/git-new-workdir. Via clever symlinking, this script creates a new working directory whose history is shared with the original repository:

$ git-new-workdir an/existing/repo new/directory

The new directory and files within can be thought of as a clone, except since the history is shared, the two trees automatically stay in sync. There’s no need to merge, push or pull.

These days, Git makes it difficult for the user to accidentally destroy data. But if you know what you are doing, you can override safeguards for common commands.

Checkout: Uncommitted changes cause checkout to fail. To destroy your changes, and checkout a given commit anyway, use the force flag:

$ git checkout -f COMMIT

On the other hand, if you specify particular paths for checkout, then there are no safety checks. The supplied paths are quietly overwritten. Take care if you use checkout in this manner.

Reset: Reset also fails in the presence of uncommitted changes. To force it through, run:

$ git reset --hard [COMMIT]

Branch: Deleting branches fails if this causes changes to be lost. To force a deletion, type:

$ git branch -D BRANCH  # instead of -d

Similarly, attempting to overwrite a branch via a move fails if data loss would ensue. To force a branch move, type:

$ git branch -M [SOURCE] TARGET  # instead of -m

Unlike checkout and reset, these two commands defer data destruction. The changes are still stored in the .git subdirectory, and can be retrieved by recovering the appropriate hash from .git/logs (see "HEAD-hunting" above). By default, they will be kept for at least two weeks.

Clean: Some git commands refuse to proceed because they’re worried about clobbering untracked files. If you’re certain that all untracked files and directories are expendable, then delete them mercilessly with:

$ git clean -f -d

Next time, that pesky command will work!

Stupid mistakes abound in the histories of many of my projects. The most frightening are missing files due to forgetting to run git add. Luckily I have yet to lose crucial data though accidental omission because I rarely delete original working directories. I typically notice the error a few commits later, so the only damage is a bit of missing history and a sheepish admission of guilt.

I also regularly commit (literally and git-erally) the lesser transgression of trailing whitespace. Though harmless, I wish these also never appeared on the public record.

In addition, though unscathed so far, I worry about leaving merge conflicts unresolved. Usually I catch them when I build a project, but there are some cases this can overlook.

If only I had bought idiot insurance by using a hook to alert me about these problems:

$ cd .git/hooks
$ cp pre-commit.sample pre-commit  # Older Git versions: chmod +x pre-commit

Now Git aborts a commit if useless whitespace or unresolved merge conflicts are detected.

For this guide, I eventually added the following to the beginning of the pre-commit hook to guard against absent-mindedness:

if git ls-files -o | grep '\.txt$'; then
  echo FAIL! Untracked .txt files.
  exit 1
fi

Several git operations support hooks; see git help hooks. One can write hooks to complain about spelling mistakes in commit messages, add new files, indent paragraphs, append an entry to a webpage, play a sound, and so on.

We encountered the post-update hook earlier when discussing Git over HTTP. This hook updates a few files Git needs for non-native communication.

How can Git be so unobtrusive? Aside from occasional commits and merges, you can work as if you were unaware that version control exists. That is, until you need it, and that’s when you’re glad Git was watching over you the whole time.

Other version control systems don’t let you forget about them. Permissions of files may be read-only unless you explicitly tell the server which files you intend to edit. The central server might be keeping track of who’s checked out which code, and when. When the network goes down, you’ll soon suffer. Developers constantly struggle with virtual red tape and bureaucracy.

The secret is the .git directory in your working directory. Git keeps the history of your project here. The initial "." stops it showing up in ls listings. Except when you’re pushing and pulling changes, all version control operations operate within this directory.

You have total control over the fate of your files because Git doesn’t care what you do to them. Git can easily recreate a saved state from .git at any time.

Most people associate cryptography with keeping information secret, but another equally important goal is keeping information safe. Proper use of cryptographic hash functions can prevent accidental or malicious data corruption.

A SHA1 hash can be thought of as a unique 160-bit ID number for every string of bytes you’ll encounter in your life. Actually more than that: every string of bytes that any human will ever use over many lifetimes.

As a SHA1 hash is itself a string of bytes, we can hash strings of bytes containing other hashes. This simple observation is surprisingly useful: look up hash chains. We’ll later see how Git uses it to efficiently guarantee data integrity.

Briefly, Git keeps your data in the ".git/objects" subdirectory, where instead of normal filenames, you’ll find only IDs. By using IDs as filenames, as well as a few lockfiles and timestamping tricks, Git transforms any humble filesystem into an efficient and robust database.

How does Git know you renamed a file, even though you never mentioned the fact explicitly? Sure, you may have run git mv, but that is exactly the same as a git rm followed by a git add.

Git heuristically ferrets out renames and copies between successive versions. In fact, it can detect chunks of code being moved or copied around between files! Though it cannot cover all cases, it does a decent job, and this feature is always improving. If it fails to work for you, try options enabling more expensive copy detection, and consider upgrading.

For every tracked file, Git records information such as its size, creation time and last modification time in a file known as the index. To determine whether a file has changed, Git compares its current stats with that held in the index. If they match, then Git can skip reading the file again.

Since stat calls are considerably faster than file reads, if you only edit a few files, Git can update its state in almost no time.

You may have been wondering what format those online Git repositories use. They’re plain Git repositories, just like your .git directory, except they’ve got names like proj.git, and they have no working directory associated with them.

Most Git commands expect the Git index to live in .git, and will fail on these bare repositories. Fix this by setting the GIT_DIR environment variable to the path of the bare repository, or running Git within the directory itself with the --bare option.

This Linux Kernel Mailing List post describes the chain of events that led to Git. The entire thread is a fascinating archaeological site for Git historians.

Here’s how to write a Git-like system from scratch in a few hours.

$ echo sweet > YOUR_FILENAME
$ git init
$ git add .
$ find .git/objects -type f

"blob" SP "6" NUL "sweet" LF

$ printf "blob 6\000sweet\n" | sha1sum

$ git cat-file -p aa823728ea7d592acc69b36875a482cdf3fd5c8d

$ git commit  # Type some message.
$ find .git/objects -type f

$ git filter-branch --tree-filter 'mv YOUR_FILENAME rose'
$ find .git/objects -type f

"tree" SP "32" NUL "100644 rose" NUL 0xaa823728ea7d592acc69b36875a482cdf3fd5c8d

$ echo 05b217bb859794d08bb9e4f7f04cbda4b207fbe9 | git cat-file --batch

$ zpipe -d < .git/objects/05/b217bb859794d08bb9e4f7f04cbda4b207fbe9 | sha1sum

$ rm -r .git/refs/original
$ git reflog expire --expire=now --all
$ git prune

$ git commit --amend -m Shakespeare  # Change the commit message.
$ git filter-branch --env-filter 'export
    GIT_AUTHOR_DATE="Fri 13 Feb 2009 15:31:30 -0800"
    GIT_AUTHOR_NAME="Alice"
    GIT_AUTHOR_EMAIL="alice@example.com"
    GIT_COMMITTER_DATE="Fri, 13 Feb 2009 15:31:30 -0800"
    GIT_COMMITTER_NAME="Bob"
    GIT_COMMITTER_EMAIL="bob@example.com"'  # Rig timestamps and authors.
$ find .git/objects -type f

"commit 158" NUL
"tree 05b217bb859794d08bb9e4f7f04cbda4b207fbe9" LF
"author Alice <alice@example.com> 1234567890 -0800" LF
"committer Bob <bob@example.com> 1234567890 -0800" LF
LF
"Shakespeare" LF