Releases

Schedule

New versions of NSCLI were typically released each month around the 15th, but due to support challenges we currently release on an as-needed basis.

Versioning

NSCLI uses an implementation of the CalVer version number standard. Our specific implementation is described by scripts/versioning.py.

Publishing

NSCLI releases are published on our github. They are also mirrored on Artifactory, which is where most users get them.

Artifactory Releases

Python:

https://artifacts.rackspace.net/artifactory/nscli-pypi-local/nscli/

Homebrew:

https://artifacts.rackspace.net/artifactory/nsi-bottles-local/

Debian:

https://artifacts.rackspace.net/artifactory/nscli-debian-local/pool/

Process

This section contains notes about our current release process, and is a working document that is subject to change at any time.

Making Releases

Prerequisites/Setup

  • Have PDM installed.

    • As of Jul 27th, 2023 - we have to stay on v2.7.4.

    • constraints.txt creation (scripts/generate_constraints.py) breaks due to missing section in pdm.lock.

    • Potential solution is to use pdm lock –static-urls. REF : https://github.com/pdm-project/pdm/pull/2101/files

  • GPG Key

    • Ensure you have a GPG key. Create one if needed.
      • gpg –list-keys (Check for gpg keys)
        • If gpg command is missing:

          • brew install gnupg

      • Create a GPG key
    • Ensure you have your gpg configured on github.
    • Ensure git is configured with your gpg key.

  • YOU MUST HAVE NSCLI TAPPED IN HOMEBREW

  • Set your system python to 3.9.X

  • Fork NSCLI and set a git remote of NSI/nscli as NSI:

    git remote add NSI git@github.rackspace.com:NSI/nscli.git

  • Ensure you have gh-pages as a local branch tracking NSI/gh-pages

    git fetch --all
git branch gh-pages NSI/gh-pages

! Validate the status of your gh-pages branch with ::

git branch -vv | grep gh-pages

! It should look like this:
    - gh-pages                                58b4c5576 **[NSI/gh-pages]** rebuild docs

  • Ensure you have master as a local branch tracking NSI/master

    git fetch --all
git branch master NSI/master

! Validate the status of your master branch with

    git branch -vv | grep master

! It should look like this:
    - master                                ccdfb27f9 **[NSI/master]** Merge pull request #2904 from NSI/develop

Few days before release

OPTIONAL - (needs more testing) Update the lockfile with latest compatible packages

deactivate
make distclean
git fetch --all
git checkout develop
git pull NSI develop
make pdm_lock

Two days before release

E-mail nscli@rackspace.com and let them know there will be a release soon, so they can finish any code they’re in the middle of.

One day before release

Perform a brew update && brew upgrade to avoid waiting for package upgrades on release day.

Release day

  1. Try to merge or reject any outstanding PRs on the development branch. If any PRs need to stay open past release, make sure the authors know about it.

  2. Pull down the development branch and build a release env for testing.:

    deactivate
make distclean
git fetch --all
git checkout develop
git pull NSI develop
git log
git status
make release_env
source .venv/bin/activate
nscli -v

  3. - SKIP - (pending tests review & validation) Run the tests. Make sure they pass.

    pytest nscli/tests

  4. Check the basic functionality of the develop branch:

    • Run nscli info on a device

    • SSH to a network bastion

    • SSH to a switch

    • SSH to a customer bastion

    • SSH to a firewall

    • SSH to an F5 or other device GUI

    Lab Devices to test against:

    • ASA 349763

    • F5 689680

    • WAF 863021

nscli info 349763
nscli netbast fra1
nscli netbast ord1
nscli switch C7-10-1E.ORD1
nscli cbast dfw1
nscli cbast syd2
nscli ssh 349763
nscli ssh 689680
nscli ssh 689680 -g
nscli ssh 863021
nscli ssh 863021 -g

  1. Clean up the release env and rebuild with a build env:

    deactivate
make distclean
make build_env
source .venv/bin/activate

  2. If you need to fix anything, merge it into the development branch and repeat steps 1-4 until everything works. You don’t need to make a PR yet. Any changes can be merged alongside later commits in this process.

  3. Make sure the nscli version is up-to-date in nscli/__init__

  4. Make sure the documentation is up-to-date in the development branch, especially Commands

    • To find what has changed since the last release, you can do:

      git log -p $(git tag | tail -n 1)..

    • see man git-log for lots of filtering options.

    • Run make webpage and take a look at the generated page; make any adjustments needed to formatting in the source docs:

      make webpage

    • Review docs/src/newsfragments. If there are any news file still in here, troubleshoot why. Maybe they have the wrong extension? Troubleshooting:

      make -q docs/src/Changelog.rst
echo $?
# 0 means up to date
touch docs/src/newsfragments/<pickone>
make -q docs/src/Changelog.rst
echo $?
# 1 means not up to date
make webpage

tl;dr:
    touch docs/src/newsfragments/<pickone>
    make webpage

    • Review HTML formatting and modify rst files as needed.

      open docs/gh-pages/index.html

    • Note: running make webpage is likely to stage a commit for the updates to the Changelog. Make sure this is committed before proceeding.

      git status
git add <filenames> (if you made additional changes)
git commit -m "Rebuilds changelog vXX.X.XX"

  5. Merge any documentation changes into the development branch. Make a PR to NSI/develop.

    git push
! create a PR from <your_fork>/develop -> NSI/develop

  6. Deactivate from any applied virtual env

    deactivate

  7. Create a pull request to merge the development branch into master. Have someone from the NSI team review, approve, and merge.

    • Title: NSCLI vYY.M.D

    • kill pr-merge jenkins job

    • wait for branch job to finish, then “build now” on cancelled pr-merge

    • merge into master

  8. Pull down master:

    git checkout master && git pull –rebase git@github.rackspace.com:NSI/nscli.git master

  9. Export the environment variables that the Makefile needs in order to proceed.:

    export API_TOKEN=<ID_TOKEN>

    # Identity token for Artifactory # Found in artifactory > username > Identity Tokens

    export OAUTHTOKEN=<TOKEN>

    # For Rackspace GitHub # Found in github > settings > developer settings > personal access tokens

  10. Clean up existing brew/nscli installation.:

    brew uninstall nscli && brew cleanup

  11. Clean up potentially old nscli docker images and login to docker.

    • Note: Login credentials will be your SSO

    • WARNING: This will DELETE ALL local images. Upload / save them before running.

    • Potentially update the command to only delete “NSCLI” images with “filter”

    docker system prune -af && docker login docker.artifacts.rackspace.net && docker login nsi-docker-local.artifacts.rackspace.net

  12. Update brew:

    brew update && brew upgrade

  13. Clean up and build a new release environment to define a list of prod packages.:

    make distclean && make release_env

  14. Create a list of prod packages in pdm_list.:

    make pdm_list

  15. Create a new ‘build’ environment used for the remaining build process.:

    make build_env

  16. Make the python sdists:

    make dist

  17. Make the final copy of webpages.:

    make webpage

  18. Upload the python sdist. This “deploys” a new nscli version.:

    ! check the changelog.html, verify all updates are present

    make upload

  19. Build the debian packages.

    make -j 3 debian

  20. Validate there are 3 debian packages built in ./dist/debian, then deploy the debian packages - Note: If there are not 3 builds, investigate errors during step 23 and ensure you delete those docker images before rebuilding in step 23.

    make debian_upload

  21. Homebrew

    1. Person1(arm64) - Build the nscli brew formula.

      make brew_formula

    2. Have Person2(x86_64) catch up with setting their build_env, then hand over the generated Formula/nscli.rb to Person2(x86_64) and place it in Formula/nscli.rb.:

      ! Person2: Make sure you are on NSI master / develop (with latest changes pulled)
! Person2: Create following directories at root (same folder as Makefile), if missing:  dist, Formula

brew uninstall nscli && brew cleanup
make distclean
make build_env
> copy Person1's(arm64 Mac) formula file to Formula/nscli.rb
> MAKE SURE PERSON2: creates a "dist" FOLDER in root nscli folder

    3. Person1 & Person2 - Make a new bottle of nscli:

      make bottle

    4. Once the bottle is built, copy the sha256 from the bottle do from the formula and have Person1 manually insert it into their formula, nscli.rb, that will be uploaded.

      Sample Complete DSL bottle do:

      bottle do
    root_url "https://artifacts.rackspace.net/artifactory/nsi-bottles-local"
    sha256 big_sur: "a3c9a00f97426019a60b37030402fc0f8968cd2500a565664b3257f59ad08cd8"
    sha256 cellar: :any, arm64_monterey: "f8627df9323b4db4c75d288610940e2dedc5424428ea94cf9391d1bcadc8e0e1"
end

    5. Making the bottle will install nscli. Open a new terminal window and make sure the installed nscli works.:

      nscli -v
nscli ssh 349763

    6. Upload the bottle to artifactory, and the new Formula to github.

      • Person1(arm64):

        make bottle_upload formula_upload

      • Person2(x86_64):

        make bottle_upload

    7. Steps for Person2(x86_64) are complete. Remaining steps are done by Person1.

    8. Upload /Formula/nscli.rb file to mirror, so it can be used for “non Tap” installation, if needed.

      • Create a version specific sub-folder under content root:

        /home/netsec.devops/content/nscli/Formula/23.7.27/

  22. Upload the new web docs with make webpage_upload and review them at https://pages.github.rackspace.com/nsi/nscli/

    make webpage_upload

  23. Ensure someone merges the changes for the uploaded formula in the SupportTools/homebrew-rs repo.

  24. Tag master with the new version. The version number format is ‘v’ + the number, like v21.1.14. Example: git tag -s vYY.M.D master. Push the signed tag to GitHub: git push NSI --tags

    git tag -s vYY.M.D master -m “NSCLI vYY.M.D” && git push NSI –tags

  25. Create a new release in GitHub. Copy/paste the release notes from docs/src/Changelog.rst into the release notes. Upload the new tarball and wheel as attachments.

    • Click Releases

    • Target: master

    • Tag: vYY.M.D

    • Title: NSCLI vYY.M.D

    • Copy/paste from docs/src/Changelog.rst

    • Drag and drop .tar.gz and .whl files from ./dist/

    • Click Publish Release

  26. Make the release email with

    make email

    This will create the release e-mail and send it to you to preview. To send the real release e-mail, run scripts/send_release_email.py. This currently doesn’t work, so just forward the test e-mail, making sure to update the subject and fix the formatting.

    • NOTE: You will need to have a sshuttle session opened to the SMTP server.
      • First add a route to smtp.ord1.corp.rackspace.com

        • sudo route add 10.12.47.12/32 -iface en0

      • Create the proxy

        • sshuttle -r cbast.dfw1.corp.rackspace.net smtp.ord1.corp.rackspace.com

      • Remove the route once you’ve received the email

        • sudo route delete 10.12.47.12/32

    • Troubleshooting:

      • make sure sso is up to date in keychain com.rackspace.nscli.sso

      • If email fails, just copy/paste from docs/email/ReleaseEmail.html

    To:

    GTS NSIA Engineers <GTS-Netsec-AE@rackspace.com>;

    CC:

    GTS Engineering Leadership <GTS-Eng-LT@rackspace.com>;
GTSNetSec <GTSNetSec@rackspace.com>;
NSI Stakeholders <nsi_stakeholders@rackspace.com>;
GTS Network Defense <GTS-Network-Defense@rackspace.com>;
NSCLI Dev <nscli@rackspace.com>;
GTS First Line Response - NetSec <GTSFLR-NETSEC@rackspace.co.uk>;
Provisioning - NetSec <provisioning-netsec@rackspace.com>
GSS Cloudflare EES <GSS-Cloudflare-EES@rackspace.com>;

    Subject: New NSCLI available: vYY.M.D

    In the body, at the bottom add:

    BCC: GTS Engineering Leadership; GTSNetSec;  NSI Stakeholders; GTS Network Defense; NSCLI Dev; GTS First Line Response - NetSec; Provisioning - NetSec; GSS Cloudflare EES;

  27. Check out develop. Bump the version number in nscli.__init__. Push these changes to GitHub.:

    git checkout develop
git log

!! update ``__init__.py``

git add nscli/__init__.py
git commit -m "version bump"
git push

!! Create and merge a PR to ``NSI/develop``