-------------------------------------------------------------------------------
General GIT notes...

-------------------------------------------------------------------------------
Push a new GIT project to a shared GIT server

Create bare GIT repo on the shared server.
   cd /share/git   # or wherever
   mkdir project
   cd project
   git --bare init
   git add -A
   git commit -a

OR
   using gitlab interface, press '+' to create the new repository

In your local repository...
NOTE you may have to 'unprotect' the repo project main branch

   cd $HOME/project   # or wherever
   # Now change the location to the bare share repository
   git remote add origin file:///share/git/project    # or wherever

   # and push to to populate the remote repository
   git branch -M main
   git push origin main

-------------------------------------------------------------------------------
Download Example...
  git clone git://github.com/jordansissel/xdotool.git
  cd xdotool

Update the download (pull updates)...
  git pull

Recursive pull

  for dir in $(find * -maxdepth 3 -type d -name '.git'); do
    repo=${dir%'/.git'};  printf "%-28s" $repo
    ( cd $repo; git pull; )
  done

  # reset to remote repo
  git reset --hard $(git log --pretty=%H | tail -n 1); git pull


Commit and upload changes (if allowed)
  git status         # How does my branch differ
  git add {file}
  git add -A         # do all adds and deletes

  git commit -a                  # commit all files (except unknwon files)
  git commit {file}              # commit changes to just this file
  git commit --reuse-message=HEAD@{1}   # reuse last message
  git commit -C HEAD@{1}                # ditto
  git commit -c ORIG_HEAD        # new commit, reuse previous message

  git commit --amend             # replace the last commit (changes or message)
                                 # do not do this if you've already pushed!

  git push               # push changes to server
  git push -o ci.skip    # push but don't run any CI pipeline

  git tag           # list tags
  git tag {NAME}    # tag the last commit
  git push --tags   # push up the tag change

  git tag -d {NAME}   # remove tag
  git tag {NAME}      # set it again
  git push --tags -f  # force it upward

  # Commits and branches of a repository
  git log --oneline
  git log --pretty=oneline --abbrev-commit
  git log --graph --oneline --date-order --decorate --color --all

  git reflog                # list commit logs

  git diff                  # What has not been commited
  git diff origin/master    # uppush changes to head of remote copy
  git diff ORIG_HEAD        # unpush changes to what was downloaded
  git diff HEAD~1           # What was just committed!
  git difftool              # graphical multi-file (see ".config/git/config")
         Example
           git diff -- config_app.pp   # uncommited changes for a file
           git diff e2468af..e9d1f29   # the changes mentioned in a pull
           git diff 5.4.3..5.5.0       # changes between two tags

  # Diff against the remote branch (what has changed between two)
  git diffuse remotes/origin/development -- {file}

  git reset --hard               # delete all work since last commit???
  git reset --soft HEAD~1        # undo last commit (leave work dir alone)
  git reset --hard HEAD~1        # undo last commit (and all changes)
                                 # NOTE: reflog will still remember!
  git revert {commit id}         # undo a commit (add a commit that undoes it)


Backtracking
  git checkout -- {file|dir}     # discard changes to {file}
  git checkout {SHA commit} -- {file|dir}     # discard changes to {file}

See who commited what line in file
  git blame {file}

Stashing

  git stash                    # save the current 'dirty' working directory
                               # so you can check something (like a pull)

  git stash list               # what have you 'stashed'
  git stash pop                # pop that last stash
  git stash apply              # ditto
  git stash apply --index      #  " also recover new 'staged' files
  git stash apply stash@{2}    # record an older stash

  git stash drop stash@{2}     # junk the saved stash

  git stash branch testchanges  # create a new branch from the stashed changes
                                # stash is then dropped

  # JUNK the 'apply'ed stash but keeping the changes you made since
  # NB: retrieving the patch of a stash and applying it in reverse
  # sort of a   "git stash unapply"
  git stash show -p | git apply -R
  git stash show -p stash@{0} | git apply -R

  # make a "git stash-unapply" command
  git config --global alias.stash-unapply '!git stash show -p | git apply -R'

# Find deleted file
# http://stackoverflow.com/a/16635324/712506
# see the changes of a file, works even if the file was deleted
  git log -- [file path]


Removing Files from Remote repository (not from local)
EG: unadd files

  # remove the files from cache (without delete)
  git rm -rf --cache  {file}
  # commit and push the change
  git commit
  git push
  # resyncronised (it isn't in cache so it should not be removed)
  git pull

  # now add the file to ".gitignore"
  vi .gitignore
     /path/to/file

Do not delete a local deletion from remote

Remove untracked files from GIT working copy
http://stackoverflow.com/questions/61212
Beware... no going back.
Use -n or --dry-run to preview the damage you'll do!
  git clean -f
  git clean -f -d # also remove directories
  git clean -f -x # also remove ignored files
  git clean -f -X # only remove ignored files


Show merge conflict file versions

  # http://www.gitguys.com/topics/merging-with-a-conflict-conflicts-and-resolutions/
  git ls-files -u
  git show :1:myfile
  git show :2:myfile
  git show :3:myfile

Diffences and Changes

  git diff               # what has changed since last commit/pull
  git diff HEAD file     # what changed in this file
  git diff --cached      # diff of last pull and the most recent commit?

  git diff HEAD^         # compared to remote head
  git diff ORIG_HEAD     # unpushed changes
  git diff development   # how has branch changed from development

  # side by side diff
  # http://stackoverflow.com/a/39225649
  git difftool -y --tool=diffuse  development
  git difftool -y -x diffuse  development


  # make a "git diffuse" command ???
  #
  git config --global alias.diffuse '!git difftool -y -x diffuse  HEAD^ | less'
  # or a simplier version, add to ".gitconfig" in  "[alias]" section.
  diffuse = difftool -y -x diffuse


  # Whole directory changes... and zoom into specific files!
  # NB: "diffuse" does not understand this, but "meld" does
  # Looks like diffuse has not been updated since 2014
  #
  git difftool --tool=meld --dir-diff development


Grep
  # configuration
  # Set line numbers
  git config --global grep.lineNumber true
  # make a little more readable
  git config --global alias.g "grep --break --heading --line-number"

  git g 'search'

  git g 'variable' -- '*.java'   # Look for 'variable' in all java files

  # look for 'arrayListName' with 'add' or 'remove' on the same line
  git g -e 'arrayListName' --and \( -e add -e remove \)

  # Google for "Git Grep Ninja"

-------------------------------------------------------------------------------
Branches

  # Clean up branches no longer in remote repository
  git remote prune origin

  # what branch are we in and are available
  git branch -a

  # switch to a different branch
  git checkout -b development
  git branch -a

  git branch | awk -F ' +' '! /\(no branch\)/ {print $2}' |
    fzf | xargs git checkout

  # create a new branch based on current, and checkout
  git checkout -b ant_dev

  # or if you have uncommitted changes...
  # move them into a new branch
  git switch -c ant_dev

  # to rename current branch (if unprotected)
  git branch -m new-branch-name

  # Rename and then Delete old remote branch
  # http://stackoverflow.com/a/4754132/712506
  git branch new-branch-name origin/old-branch-name
  git push origin new-branch-name
  git push origin :old-branch-name

  # Work on the branch, and commit
  git commit -a -m "Initial module branch"
  git push --set-upstream origin ant_dev    # first time only
  #git push                                 # later pushes

  # switch to master -- optional
  git checkout master
  git branch -a
  git pull

  # switch to my working branch
  git checkout ant_dev
  git branch -a

  git diff  development   # how has it changed from development

  # Delete a branch
  git checkout some_other_branch
  git branch -D ant_dev
  git branch -a
  git remote prune origin

-------------------------------------------------------------------------------
Seperate worktree

  https://git-scm.com/docs/git-worktree

These are branches checked out at another location at the same time as the
original branch.  Very suitable for working on a branch (or a throwaway)
without distrubing the normal development.

  git worktree add -d ../hotfix
        creates a new branch named 'hotfix' at the location specified

  git worktree list
        list worktrees attachedto this checkout

  git worktree remove
        remove the worktree from the repository

-------------------------------------------------------------------------------
Cherry-Pick and File Merges

Cherry pick
This is Best done in gitlab, unless you want to cheery pick multiple commits

  git log                       # get the long commit number you want to merge
  git checkout production       # go to branch you want to merge into
  git pull                      # and that it is up to date


  # commit (the -x creates a standard message)
  # (use -e to edit message before hand)
  git cherry-pick -x {number}...

  git push origin production    # push local repository to remote


Merge a whole file from one branch to another

  git checkout {destination_branch}
  git pull
  # diff it   (WARNING: the diff is reversed!)
  git diff {source_branch} -- {files}...

  # EITHER copy the file over
  git checkout {source_branch} -- {files}...

  # OR interactive selection of the differences
  git checkout --patch {source_branch} -- {files}...

  # commit
  git commit -m "Cherry pick '{file}' from {source} branch"

  # What changed
  git diff remotes/origin/{destination_branch}

  git push

-------------------------------------------------------------------------------
Forking and Merging from other people

Create a fork...
  git clone {url_of_project_to_fork}

Make a branch of the 'upstream' project
  git remote add upstream {url_of_project_to_fork}

Set the remote branches
  git branch -a | cat
  git remote -v

Remotes are just branches where you can fetch changes from the
remote server, before merging tem into your master or some branch.

Fetch changes
  git fetch upstream

merge their changes to yours
  git checkout master
  git merge upstream/master

Merge changes in a fork back to the upstream (if you have access)
  git push upstream/master

If you don't have access ask the maintainer to create a pull request...

-------------------------------------------------------------------------------
Fetch, Merge, Pull, Rebase

NOTE: a 'pull' is actually the same as a 'fetch' then 'merge'
So you could do both at the same time. BUt if you don't you can
  git fetch upstream
  git diff master upstream/master
When you are satified THEN you can do the merge
  git merge upstream/master


Discard all commits not yet pushed...
From  https://stackoverflow.com/questions/4785107/

  git fetch
  git reset --hard origin/master

---
'rebase' will merge changes into a branch, but then change the 'base' of the
branch to match where those changes come from.

See   https://www.atlassian.com/git/tutorials/merging-vs-rebasing

Merge the last three commits into a single commit
This is done before pushing, interactiveally
  git rebase -i HEAD~3

Do NOT do this if you have already pushed!

---
'filter-branch' can be used to remove a file from ALL the commits,
ssay one that contains a password...

  git filter-branch --tree-filter 'rm -f passwords.txt' HEAD

Make a subdirectory the 'new root' for every commit,
It will remove commits that did not effect the sub-directory.

  git filter-branch --subdirectory-filter sub-dir HEAD

Change your email in all commits....

  git filter-branch --commit-filter '
        if [ "$GIT_AUTHOR_EMAIL" = "schacon@localhost" ];
        then
                GIT_AUTHOR_NAME="Scott Chacon";
                GIT_AUTHOR_EMAIL="schacon@example.com";
                git commit-tree "$@";
        else
                git commit-tree "$@";
        fi' HEAD

---
BFG  (alternative to 'git filter-branch')
Good to remove Files, Passwords, Private data

  Clone a 'bare' copy of repo...
    git clone --mirror git://example.com/some-big-repo.git
  and make a backup
    java -jar bfg.jar --strip-blobs-bigger-than 100M some-big-repo.git
    cd some-big-repo.git
    git reflog expire --expire=now --all && git gc --prune=now --aggressive
    git push


  bfg --strip-blobs-bigger-than 100M --replace-text banned.txt repo.git

-------------------------------------------------------------------------------
clean out old large files from git history

https://web.archive.org/web/20190207210108/
  http://stevelorek.com/how-to-shrink-a-git-repository.html

Clone the repository
  git clone remote-url

Get a deep clone (all branches)
  for b in `git branch -a | grep remotes | grep -v HEAD | grep -v master`; do
    git branch --track ${b##*/} $b
  done


List the large files in the repository
  File: git_large_files.sh

=======8<--------CUT HERE----------
#!/bin/bash
#set -x

# Shows you the largest objects in your repo's pack file.
# Written for osx.
#
# see http://stubbisms.wordpress.com/2009/07/10/
#        git-script-to-show-largest-pack-objects-and-trim-your-waist-line/
# author Antony Stubbs

# set the internal field spereator to line break, so that we can
# iterate easily over the verify-pack output
IFS=$'\n';

# list all objects including their size, sort by size, take top 10
objects=`git verify-pack -v .git/objects/pack/pack-*.idx |
            grep -v chain | sort -k3nr | head`

echo "'pack' lists sizes of the object, compressed, inside the pack file."
echo "All sizes are in kB."

output="size,pack,SHA,location"
for y in $objects; do
  # extract the size in bytes
  size=$((`echo $y | cut -f 5 -d ' '`/1024))

  # extract the compressed size in bytes
  compressedSize=$((`echo $y | cut -f 6 -d ' '`/1024))

  # extract the SHA
  sha=`echo $y | cut -f 1 -d ' '`

  # find the objects location in the repository tree
  other=`git rev-list --all --objects | grep $sha`

  #lineBreak=`echo -e "\n"`
  output="${output}\n${size},${compressedSize},${other}"
done

echo -e $output | column -t -s ',

=======8<--------CUT HERE----------

clean a specific file
  git filter-branch --tag-name-filter cat \
           --index-filter 'git rm -r --cached --ignore-unmatch {filename}' \
           --prune-empty -f -- --all

Now remove it from the history
  rm -rf .git/refs/original/
  git reflog expire --expire=now --all
  git gc --prune=now
  git gc --aggressive --prune=now

Now push it back up

  git push origin --force --all

Push the re-written tags

  git push origin --force --tags

WARNING: anyone else using the repository will need to run

  git rebase

-------------------------------------------------------------------------------
Reduce Repository Size


NOTE: you will need to 'unprotect' a branch in a gitlab or github repository

  The main branch must be unprotected
    Git Repo -> Settings -> General -> Advanced (right to the bottom)
      Repository Protection

  To delete a repository
    Git Repo -> Settings -> General -> Advanced (right to the bottom)
      -> Delete Repository

Clean out all GIT history

  This does not work completely but does remove all objects

    rm -rf .git/{HEAD,objects,info,logs}  # not refs
    git init
    git push --force

Remove all old files from repo

   du -sh .git .     # reports what is in ".git" and what isn't in ".git"
   git reflog expire --all --expire=now
   git gc --prune=now --aggressive
   git push
   du -sh .git .

Remove all but last N commits (N=1 for no history at all)

  # list the log
  git log --oneline

  # get commit commit number to rebase to
  n=1    # the very last commit!
  commit=$(git log --oneline | sed -n "${n}p" | cut -f1 -d\  )

  # check that commit as a ophan branch
  git checkout --orphan temp $commit --force
  git commit -m 'rebase commit'

  # rebase to make it the new start commit
  git rebase --onto temp $commit master

  # clean up
  git branch -D temp
  git checkout master

  # Now 'unprotect' the master branch on the Git Server
  #   repo -> settings -> Repository -> Protected Branches

  # push it to the repository
  git push -f origin master

  # You can now protect your master branch again (if desired)

  # and any old files
  rm -Rf .git/refs/original
  rm -Rf .git/logs/
  git gc
  git prune --expire now


Rewrite (Clear) GIT History

  Every now and then you need to rewrite history in a Git repo, be it to remove
  passwords or some other sensitive data.

  Thankfully a tool exists to do most of the heavy listing for us, it's
  a fairly simple process

    echo 'PASSWORD_GOES_HERE' > pw.txt

  Set the content to be removed, BFG also handles globbing and regexes
    docker run -it --rm --user $USER --volume "$PWD":/data \
          --workdir /data \
          --replace-text pw.txt --private \
          soodesune/bfg-repo-cleaner

  Run BFG across the repo, wipes out commit hashes. We use a containered
  version here to save installation/setup

    git reflog expire --expire=now --all

  Tell git to redo all the logs

    git gc --prune=now --aggressive

  Tell git to delete any commits without a proper reference

    git push

  Override the remote git repo, might need to unprotect the branches for this
  operation

  If you simply reset the git history and push then none of the information is
  really reset - you need to leave the commits but rewrite their content in
  order to properly revoke data from git.

-------------------------------------------------------------------------------
Self signed certificate server

This stops errors such as "error setting certificate verify locations"

  git config --global http.sslverify false

-------------------------------------------------------------------------------
SSH Signed GIT commits (prove you are the committer)

~/.ssh/gitkeys

  a.roffey@griffith.edu.au ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFbmKlm7c4OuvXMnmN3gg0+azB7vJ7Cca9X2j4klcidG 20210908@worklaptop

~/.gitconfig

  [commit]
  gpgsign=true

  [user]
  name = Andrew Roffey
  email = a.roffey@griffith.edu.au
  signingKey=~/.ssh/id_ed25519

  [gpg]
  format=ssh

  [gpg.ssh]
  allowedSignersFile=~/.ssh/gitkeys

-------------------------------------------------------------------------------
3rd party tools - git command

Git will call any executable found on the path that begins with "git-",
so if you create a shell script called "/usr/bin/git-foo"
then you can do "git foo".

Gitolite
  http://gitolite.com/gitolite/index.html

Simplier, more features.  GitLab uses gitolite underneath.

Gerrit
  https://code.google.com/p/gerrit/

Gerrit is used for Code review, but you can actually use it also for managing
users and give them good defined permissions. You can bypass code-review
(trough access controls) and use it just for managing projects and ssh-keys.
Gerrit has a really strong access control mechanism.

-------------------------------------------------------------------------------