python commit and push to git in Jenkins

I’ve looked all over the internet and can’t find anything regarding pushing to a remote repo via python…. particularly in Jenkins where your upstream is not set.

So I finally found the only helpful link and it was saying: don’t use gitpython.

But after trying multiple methods, I finally got this working. Note: It uses a few args from Jenkins && the point of this script is nothing more than to modify a file and commit it. With this functionality, I can now do real work.

import git
import os
import time
import yaml
import sys
import re
from git import Repo
print os.environ["GIT_BRANCH"]
workspace=os.environ['WORKSPACE']
m = re.search("origin/(.*)", os.environ["GIT_BRANCH"])
if m:
    git_branch = m.group(1)
else:
    sys.exit("COULD NOT LOAD SOURCE BRANCH")
# UPDATE VERSION FILE
with open(workspace + '/deploy_automation/config/int/versions.yaml', 'r') as f:
    versions_yaml = yaml.load(f)
versions_yaml["a.component"] = time.time()
with open(workspace + '/deploy_automation/config/int/versions.yaml', 'w') as f:
    yaml.dump(versions_yaml, f, default_flow_style=False)
# OTHER WAY
git_repo = Repo(workspace + "/deploy_automation")
git_repo.git.status()
git_repo.git.add(workspace + '/deploy_automation/config/int/versions.yaml')
git_repo.git.config('--global', "user.name", "user name")
git_repo.git.config('--global', "user.email", "user@domain.com")
git_repo.git.status()
git_repo.git.commit(m=' DEPLOY SCRIPT Updating versions.yaml for ENV jamestest2 and Service test')
git_repo.git.push('--set-upstream', 'origin', git_branch)

					

git checkout branch

  1. Clone repo

    git clone git@gitlab.sandlininc.com:product/component.git
  2. List Branches

    git branch -a
    * master
      remotes/origin/HEAD -> origin/master
      remotes/origin/add_javadocs
      remotes/origin/implementation-1.0.7
      remotes/origin/implementation-1.0.8
      remotes/origin/implementation-1.0.x
      remotes/origin/implementation-2.0
      remotes/origin/implementation-2.0.0
      remotes/origin/master
      remotes/origin/product-3.5
      remotes/origin/product-4.0
      remotes/origin/trunk
    
  3. Switch to branch
    git checkout -b product-3.5
    Switched to a new branch 'product-3.5'
    

List tags in GIT

List tags in repo

box:product user$ git tag
product-0.3
product-0.4
...
product-1.7
product-1.8

With a little more detail:

box:product user$ git tag -l -n1
product-0.3 [maven-release-plugin]  copy for tag product-0.3
product-0.4 [maven-release-plugin]  copy for tag product-0.4
...
product-1.7 [maven-release-plugin]  copy for tag product-1.7
product-1.8 [maven-release-plugin]  copy for tag product-1.8

With more detail:

box:product user$ for t in `git tag -l`; do git cat-file -p `git rev-parse $t`; done
object 965d40c9c8091c73871761034b78135793368fa0
type commit
tag product-0.3
tagger company <devops@company.com> Mon Oct 7 11:57:00 2013 -0700

[maven-release-plugin]  copy for tag product-0.3
object bff67daf9605abd87adb1122ea4e9081172ab99e
type commit
tag product-0.4
tagger company <devops@company.com> Mon Oct 7 12:26:45 2013 -0700

...
[maven-release-plugin]  copy for tag product-1.7
object 6a9cbd9107c1ea18a9b9b540b19c93a57f0b515d
type commit
tag product-1.8
tagger company <devops@company.com> Tue Dec 3 11:19:43 2013 -0800

Delete tag in git

Delete a tag

box:product user$ git tag -d product-1.8
Deleted tag 'product-1.8' (was 2c41c34)
box:product user$ git push origin :refs/tags/product-1.8
To git@gitlab.companycorp.com:product2/uikit.git
 - [deleted]         product-1.8

Changing a remote’s URL

source

git remote -v
# View existing remotes
# origin  https://github.com/user/repo.git (fetch)
# origin  https://github.com/user/repo.git (push)

git remote set-url origin https://github.com/user/repo2.git
# Change the 'origin' remote's URL

git remote -v
# Verify new remote URL
# origin  https://github.com/user/repo2.git (fetch)
# origin  https://github.com/user/repo2.git (push)

SVN incoming edit / delete upon merge

  1. Sometimes when merging in svn, a file has been edited / deleted in both locations. To resolve:
  2. May look something like:
    user@boxname:/opt/repo/deploy$ svn status
    M .
    ? target
    ! C deploy
    > local delete, incoming edit upon merge
    ! C foo.py
    > local delete, incoming delete upon merge
    D xxx.yml
    C bar.py
    > local add, incoming add upon merge
  3. To resolve, execute for each line:
    $ svn resolve --accept working ${filename}
  4. EX:
    $ svn resolve --accept working deploy
    $ svn resolve --accept working foo.py
    $ svn resolve --accept working bar.py

svn: Can’t open file ‘${repo}/db/txn-current-lock’: Permission denied

Trying to do a SVN commit & getting:

user:workspaces user$ svn checkout http://svn.test.com/core
Checked out revision 0.
user@desktop$ cd core
user@desktop$ touch foo
user@desktop$ svn add foo
A         foo
user@desktop$ svn commit -m "test"
svn: Commit failed (details follow):
svn: Can't open file '/opt/cmtools/svn_repos/core/db/txn-current-lock': Permission denied

http://stackoverflow.com/questions/960241/svn-permission-denied
Turns out it’s a selinux error

root@server$ chcon -R -t httpd_sys_content_rw_t /opt/cmtools/svn_repos/core/
user@desktop$ svn commit -m "test"
Adding         foo
Transmitting file data .
Committed revision 1.

SVN Revert codebase to a previous revision

  1. Figure out revision number of old good version
    svn log http://svn.domain.com/app/branches/3_3_0 -r {2010-09-15}:{2010-09-20}
    
  2. See what code will be reverted
    svn merge --dry-run -r 25951:25950  http://svn.domain.com/app/branches/3_3_0
    
  3. Perform the revision
    svn merge -r 25951:25950  http://svn.domain.com/app/branches/3_3_0
    
  4. Commit your change
    svn commit -m "Description"
    

Merge all changes from branch to trunk while leaving merge conflicts up to developers

  1. This document is assuming you have already checked out & updated trunk.
    svn checkout http://server/svn/project/trunk ~/workspaces/project_trunk
    svn update ~/workspaces/project_trunk
  2. Check out the branch to merge from
    svn checkout http://server/svn/project/branches/0.0.6 ~/workspaces/project_0.0.6
  3. Figure out the revision the branch began
    $ svn log --verbose --stop-on-copy http://server/svn/project/branches/0.0.6
    ------------------------------------------------------------------------
    r3 | joe | 2010-08-03 12:50:08 -0700 (Tue, 03 Aug 2010) | 1 line
    Changed paths:
       A /branches/0.0.6/test
       A /branches/0.0.6/global
    
    Adding dirs for test sprint 1
    ------------------------------------------------------------------------
    r2 | joe | 2010-08-03 12:49:12 -0700 (Tue, 03 Aug 2010) | 1 line
    Changed paths:
       A /branches/0.0.6 (from /trunk:1)
    
    Creating branch for test Sprint 1
    ------------------------------------------------------------------------

    NOTE: The revision where the branch was created ($BCV = 2)

  4. CD into the trunk you are merging to & get the revision num.
    cd /workspaces/project_trunk
    svn update

    This will return a message “At revision XXXXX”

    jsmith@PC-JSMITH /workspaces/project_trunk
    $ svn update
    At revision 92.

    $TV=92

  5. CD to your trunk which you are merging TO
    $ cd /workspaces/project_trunk
  6. Merge the changes from the branch to the trunk
    $ svn merge --accept postpone -r ${BCV}:${TV} http://server/svn/project/branches/0.0.6
    --- Merging r2 through r92 into '.':
    C    test-3pom.xml
    Skipped 'foo.bar'
    C    test-parentpom.xml
    U    test-1srcmainjavacomarrisilibTest.java
    U    test-1srcmainjavacomarrisilibApp.java
    C    test-1pom.xml
    C    test-2pom.xml
    Summary of conflicts:
    Text conflicts: 4
    Skipped paths: 1
  7. Generate a report of merge conflicts
    cd /workspaces/project_trunk
    find . -type f -name "*.working"

    EX:

    jsmith@PC-JSMITH /workspaces/project_trunk/trunk
    $ find . -type f -name "*.working"
    ./test-1/pom.xml.working
    ./test-2/pom.xml.working
    ./test-3/pom.xml.working
    ./test-parent/pom.xml.working

    Copy this list into an email & remove working from the end of each line.

  8. Get rid of all conflict resolution files
    find . -type f -name "*.working" -exec rm {} ;
    find . -type f -name "*-left.r*" -exec rm {} ;
    find . -type f -name "*-left.l*" -exec rm {} ;
  9. Commit your changes
    svn commit -m "EM-300 - Merge project 0.0.6 to trunk"
  10. Send email to DEV team to handle merge conflicts