Bash Tutorial
A brief overview of Bash, on your way to becoming a Linux expert. When a computer boots up, a kernel (MacOS, Windows, Linux) is started. This kernel provides a shell that allows user to interact with a most basic set of commands. Typically, the casual user will not interact with the shell as a Desktop User Interface is started by the computer boot up process. To activate a shell directly, users will run a “terminal” through the Desktop. VS Code provides ability to activate “terminal” while in the IDE.
Prerequisites
Setup bash shell dependency variables for this page.
- Hack: Change variables to match your project.
%%script bash
# Dependency Variables, set to match your project directories
cat <<EOF > /tmp/variables.sh
export project_dir=$HOME/vscode # change vscode to different name to test git clone
export project=\$project_dir/teacher # change teacher to name of project from git clone
export project_repo="https://github.com/nighthawkcoders/teacher.git" # change to project of choice
EOF
%%script bash
# Extract saved variables
source /tmp/variables.sh
# Access the variables
echo "Project dir: $project_dir"
echo "Project: $project"
echo "Repo: $project_repo"
Project dir: /Users/johnmortensen/vscode
Project: /Users/johnmortensen/vscode/teacher
Repo: https://github.com/nighthawkcoders/teacher.git
Setup Project
Pull code from GitHub to your machine. This script will create a project directory and add “project” from GitHub to the vscode directory. There is conditional logic to make sure that clone only happen if it does not (!) exist.
%%script bash
# Extract saved variables
source /tmp/variables.sh
echo "Using conditional statement to create a project directory and project"
cd ~ # start in home directory
# Conditional block to make a project directory
if [ ! -d $project_dir ]
then
echo "Directory $project_dir does not exists... makinng directory $project_dir"
mkdir -p $project_dir
fi
echo "Directory $project_dir exists."
# Conditional block to git clone a project from project_repo
if [ ! -d $project ]
then
echo "Directory $project does not exists... cloning $project_repo"
cd $project_dir
git clone $project_repo
cd ~
fi
echo "Directory $project exists."
Using conditional statement to create a project directory and project
Directory /Users/johnmortensen/vscode exists.
Directory /Users/johnmortensen/vscode/teacher exists.
Look at files Github project
All computers contain files and directories. The clone brought more files from cloud to your machine. Review the bash shell script observe the commands that show and interact with files and directories.
- “ls” lists computer files in Unix and Unix-like operating systems
- “cd” offers way to navigate and change working directory
- “pwd” print working directory
- “echo” used to display line of text/string that are passed as an argument
%%script bash
# Extract saved variables
source /tmp/variables.sh
echo "Navigate to project, then navigate to area wwhere files were cloned"
cd $project
pwd
echo ""
echo "list top level or root of files with project pulled from github"
ls
Navigate to project, then navigate to area wwhere files were cloned
/Users/johnmortensen/vscode/teacher
list top level or root of files with project pulled from github
Gemfile
Gemfile.lock
LICENSE
Makefile
README.md
_config.yml
[34m_data[m[m
[34m_includes[m[m
[34m_layouts[m[m
[34m_notebooks[m[m
[34m_posts[m[m
[34m_site[m[m
csa.md
csp.md
csse.md
[34mimages[m[m
index.md
indexBlogs.md
nohup.out
[34mscripts[m[m
Look at file list with hidden and long attributes
Most linux commands have options to enhance behavior
%%script bash
# Extract saved variables
source /tmp/variables.sh
echo "Navigate to project, then navigate to area wwhere files were cloned"
cd $project
pwd
echo ""
echo "list all files in long format"
ls -al # all files -a (hidden) in -l long listing
Navigate to project, then navigate to area wwhere files were cloned
/home/xavier
list all files in long format
total 80
drwxr-x--- 11 xavier xavier 4096 Aug 22 14:01 .
drwxr-xr-x 3 root root 4096 Aug 17 13:30 ..
-rw------- 1 xavier xavier 7798 Aug 22 13:26 .bash_history
-rw-r--r-- 1 xavier xavier 220 Aug 17 13:30 .bash_logout
-rw-r--r-- 1 xavier xavier 4185 Aug 22 13:23 .bashrc
drwxr-xr-x 3 xavier xavier 4096 Aug 18 14:09 .bundle
drwx------ 2 xavier xavier 4096 Aug 17 13:30 .cache
drwxr-xr-x 11 xavier xavier 4096 Aug 20 10:34 .gem
-rw-r--r-- 1 xavier xavier 94 Aug 20 11:13 .gitconfig
drwxr-xr-x 5 xavier xavier 4096 Aug 22 14:01 .ipython
-rw------- 1 xavier xavier 20 Aug 22 13:11 .lesshst
drwxr-xr-x 3 xavier xavier 4096 Aug 20 10:34 .local
-rw-r--r-- 1 xavier xavier 0 Aug 22 05:40 .motd_shown
-rw-r--r-- 1 xavier xavier 807 Aug 17 13:30 .profile
-rw-r--r-- 1 xavier xavier 0 Aug 17 13:47 .sudo_as_admin_successful
drwxr-xr-x 5 xavier xavier 4096 Aug 17 14:29 .vscode-server
-rw-r--r-- 1 xavier xavier 183 Aug 17 14:28 .wget-hsts
drwxr-xr-x 10 xavier xavier 4096 Aug 21 13:42 gems
drwx------ 3 xavier xavier 4096 Aug 18 13:27 snap
drwxr-xr-x 4 xavier xavier 4096 Aug 18 14:07 vscode
bash: line 3: /tmp/variables.sh: No such file or directory
%%script bash
# Extract saved variables
source /tmp/variables.sh
echo "Look for posts"
export posts=$project/_posts # _posts inside project
cd $posts # this should exist per fastpages
pwd # present working directory
ls -l # list posts
Look for posts
/home/xavier/vscode/student/_notebooks
total 52
-rw-r--r-- 1 xavier xavier 31894 Aug 22 14:05 2023-08-16-linux_shell.ipynb
-rw-r--r-- 1 xavier xavier 5415 Aug 17 13:41 2023-08-17-AP-pseudo-vs-python.ipynb
-rw-r--r-- 1 xavier xavier 8615 Aug 17 13:41 2023-08-21-VSCode-GitHub_Pages.ipynb
bash: line 3: /tmp/variables.sh: No such file or directory
bash: line 7: cd: /_posts: No such file or directory
%%script bash
# Extract saved variables
source /tmp/variables.sh
echo "Look for notebooks"
export notebooks=$project/_notebooks # _notebooks is inside project
cd $notebooks # this should exist per fastpages
pwd # present working directory
ls -l # list notebooks
Look for notebooks
/home/xavier/vscode/student/_notebooks
total 52
-rw-r--r-- 1 xavier xavier 32192 Aug 22 14:01 2023-08-16-linux_shell.ipynb
-rw-r--r-- 1 xavier xavier 5415 Aug 17 13:41 2023-08-17-AP-pseudo-vs-python.ipynb
-rw-r--r-- 1 xavier xavier 8615 Aug 17 13:41 2023-08-21-VSCode-GitHub_Pages.ipynb
bash: line 3: /tmp/variables.sh: No such file or directory
bash: line 7: cd: /_notebooks: No such file or directory
%%script bash
# Extract saved variables
source /tmp/variables.sh
echo "Look for images in notebooks, print working directory, list files"
cd $notebooks/images # this should exist per fastpages
pwd
ls -l
Look for images in notebooks, print working directory, list files
/Users/johnmortensen/vscode/teacher/_notebooks
bash: line 6: cd: /images: No such file or directory
total 344
-rw-r--r-- 1 johnmortensen staff 37377 Jun 11 10:35 2023-05-30-linux_shell.ipynb
-rw-r--r-- 1 johnmortensen staff 7706 Jun 11 09:04 2023-05-31-VSCode-GitHub-project.ipynb
-rw-r--r-- 1 johnmortensen staff 9292 Jun 11 08:47 2023-06-01-javascript-input.ipynb
-rw-r--r-- 1 johnmortensen staff 11477 Jun 11 04:39 2023-06-01-python_hello.ipynb
-rw-r--r-- 1 johnmortensen staff 44353 Jun 11 08:06 2023-06-02-javascript_output.ipynb
-rw-r--r-- 1 johnmortensen staff 15666 Jun 11 08:48 2023-06-03-javascript_api.ipynb
-rw-r--r-- 1 johnmortensen staff 20550 Jun 11 09:06 2023-06-04-AWS-deployment.ipynb
-rw-r--r-- 1 johnmortensen staff 16271 Jun 11 04:39 2023-06-08-JWT-python.ipynb
Look inside a Markdown File
“cat” reads data from the file and gives its content as output
%%script bash
# Extract saved variables
source /tmp/variables.sh
echo "Navigate to project, then navigate to area wwhere files were cloned"
cd $project
echo "show the contents of README.md"
echo ""
cat README.md # show contents of file, in this case markdown
echo ""
echo "end of README.md"
Navigate to project, then navigate to area wwhere files were cloned
show the contents of README.md
## Blog site using GitHub Pages and Jekyll
> This site is intended for Teachers. This is to build lessons and distribute across different sections.
- This support 3 computer science sections that are in a pathway (JavaScript, Python/Flask, Java/Spring)
- JavaScript documents are new material for entry class into the pathway, they are prerequisites for the Python and Java classes.
- All course material works off of Notebooks using Python kernel, except Java which requires it own kernel.
## Preview Site
> GitHub Pages development is optimized by testing and developing on your local machine. This is called previewing you work, prior to commit and push.
- GitHub setup for, [Testing your GitHub Pages site locally with Jekyll](https://docs.github.com/en/pages/setting-up-a-github-pages-site-with-jekyll/testing-your-github-pages-site-locally-with-jekyll). After requirements are met for Jekyll and Ruby you need to install requirements for project.
```bash
bundle install
```
- Now the project is ready for preview. To simplify typing and sharing logging the details for running have be place in a ```Makefile```
- run preview server
```bash
make
```
- stop preview server
```bash
make stop
```
- test notebook conversions
```bash
make convert
```
- clean constructed files
```bash
make clean
```
end of README.md
Env, Git and GitHub
Env(ironment) is used to capture things like path to Code or Home directory. Git and GitHub is NOT Only used to exchange code between individuals, it is often used to exchange code through servers, in our case deployment for Website. All tools we use have a behind the scenes hav relationship with the system they run on (MacOS, Windows, Linus) or a relationship with servers which they are connected to (ie GitHub). There is an “env” command in bash. There are environment files and setting files (.git/config) for Git. They both use a key/value concept.
- “env” show setting for your shell
- “git clone” sets up a director of files
- “cd $project” allows user to move inside that directory of files
- “.git” is a hidden directory that is used by git to establish relationship between machine and the git server on GitHub.
%%script bash
# This command has no dependencies
echo "Show the shell environment variables, key on left of equal value on right"
echo ""
env
Show the shell environment variables, key on left of equal value on right
VSCODE_CLI=1
NIX_PROFILES=/nix/var/nix/profiles/default /Users/johnmortensen/.nix-profile
VSCODE_CRASH_REPORTER_PROCESS_TYPE=extensionHost
TERM_PROGRAM=Apple_Terminal
TERM=xterm-color
SHELL=/bin/bash
CLICOLOR=1
VSCODE_CRASH_REPORTER_SANDBOXED_HINT=1
TMPDIR=/var/folders/_n/7k1b0n557ng0fkmw74pd5cx00000gn/T/
CONDA_SHLVL=1
PYTHONUNBUFFERED=1
TERM_PROGRAM_VERSION=447
CONDA_PROMPT_MODIFIER=(base)
ORIGINAL_XDG_CURRENT_DESKTOP=undefined
MallocNanoZone=0
PYDEVD_USE_FRAME_EVAL=NO
TERM_SESSION_ID=9056DE5B-8519-46AC-8C0A-42B5FFE425B9
PYTHONIOENCODING=utf-8
USER=johnmortensen
COMMAND_MODE=unix2003
CONDA_EXE=/Users/johnmortensen/opt/anaconda3/bin/conda
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.htDuJuikD5/Listeners
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
PAGER=cat
ELECTRON_RUN_AS_NODE=1
VSCODE_AMD_ENTRYPOINT=vs/workbench/api/node/extensionHostProcess
_CE_CONDA=
CONDA_ROOT=/Users/johnmortensen/opt/anaconda3
PATH=/Users/johnmortensen/opt/anaconda3/bin:/opt/local/bin:/opt/local/sbin:/Users/johnmortensen/opt/anaconda3/bin:/Users/johnmortensen/opt/anaconda3/condabin:/Users/johnmortensen/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
LaunchInstanceID=06C30390-0D51-4B11-A295-4C30C5E96711
__CFBundleIdentifier=com.microsoft.VSCode
CONDA_PREFIX=/Users/johnmortensen/opt/anaconda3
PWD=/Users/johnmortensen/vscode/teacher/_notebooks
VSCODE_HANDLES_UNCAUGHT_ERRORS=true
ELECTRON_NO_ATTACH_CONSOLE=1
MPLBACKEND=module://matplotlib_inline.backend_inline
LANG=en_US.UTF-8
XPC_FLAGS=0x0
NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt
FORCE_COLOR=1
_CE_M=
XPC_SERVICE_NAME=0
SHLVL=5
HOME=/Users/johnmortensen
APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL=1
VSCODE_NLS_CONFIG={"locale":"en-us","osLocale":"en-us","availableLanguages":{},"_languagePackSupport":true}
PYDEVD_IPYTHON_COMPATIBLE_DEBUGGING=1
LOGNAME=johnmortensen
CONDA_PYTHON_EXE=/Users/johnmortensen/opt/anaconda3/bin/python
VSCODE_IPC_HOOK=/Users/johnmortensen/Library/Application Support/Code/1.78-main.sock
VSCODE_CODE_CACHE_PATH=/Users/johnmortensen/Library/Application Support/Code/CachedData/b3e4e68a0bc097f0ae7907b217c1119af9e03435
CLICOLOR_FORCE=1
CONDA_DEFAULT_ENV=base
VSCODE_PID=2146
GIT_PAGER=cat
VSCODE_L10N_BUNDLE_LOCATION=
VSCODE_CWD=/Users/johnmortensen/vscode/teacher
SECURITYSESSIONID=186a6
_=/usr/bin/env
%%script bash
# Extract saved variables
source /tmp/variables.sh
cd $project
echo ""
echo "show the secrets of .git"
cd .git
ls -l
echo ""
echo "look at config file"
cat config
show the secrets of .git
total 72
-rw-r--r-- 1 johnmortensen staff 9 Jun 11 09:09 COMMIT_EDITMSG
-rw-r--r-- 1 johnmortensen staff 102 Jun 11 09:09 FETCH_HEAD
-rw-r--r-- 1 johnmortensen staff 21 Jun 11 04:39 HEAD
-rw-r--r-- 1 johnmortensen staff 41 Jun 11 09:09 ORIG_HEAD
drwxr-xr-x 2 johnmortensen staff 64 Jun 11 04:39 [34mbranches[m[m
-rw-r--r-- 1 johnmortensen staff 312 Jun 11 04:39 config
-rw-r--r-- 1 johnmortensen staff 73 Jun 11 04:39 description
drwxr-xr-x 13 johnmortensen staff 416 Jun 11 04:39 [34mhooks[m[m
-rw-r--r-- 1 johnmortensen staff 4695 Jun 11 09:09 index
drwxr-xr-x 3 johnmortensen staff 96 Jun 11 04:39 [34minfo[m[m
drwxr-xr-x 4 johnmortensen staff 128 Jun 11 04:39 [34mlogs[m[m
drwxr-xr-x 34 johnmortensen staff 1088 Jun 11 09:09 [34mobjects[m[m
-rw-r--r-- 1 johnmortensen staff 112 Jun 11 04:39 packed-refs
drwxr-xr-x 5 johnmortensen staff 160 Jun 11 04:39 [34mrefs[m[m
look at config file
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
ignorecase = true
precomposeunicode = true
[remote "origin"]
url = https://github.com/nighthawkcoders/teacher.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
remote = origin
merge = refs/heads/main
Student Request - Make a file in Bash
This example was requested by a student (Jun Lim, CSA). The request was to make jupyer file using bash, I adapted the request to markdown. This type of thought will have great extrapolation to coding and possibilities of using List, Arrays, or APIs to build user interfaces. JavaScript is a language where building HTML is very common.
To get more interesting output from terminal, this will require using something like mdless (https://github.com/ttscoff/mdless). This enables see markdown in rendered format.
- On Desktop Install PKG from MacPorts
- In Terminal on MacOS
- Install ncurses
gem install mdless
Output of the example is much nicer in “jupyter”
%%script bash
# This example has error in VSCode, it run best on Jupyter
cd /tmp
file="sample.md"
if [ -f "$file" ]; then
rm $file
fi
tee -a $file >/dev/null <<EOF
# Show Generated Markdown
This introductory paragraph and this line and the title above are generated using tee with the standard input (<<) redirection operator.
- This bulleted element is still part of the tee body.
EOF
echo "- This bulleted element and lines below are generated using echo with standard output (>>) redirection operator." >> $file
echo "- The list definition, as is, is using space to seperate lines. Thus the use of commas and hyphens in output." >> $file
actions=("ls,list-directory" "cd,change-directory" "pwd,present-working-directory" "if-then-fi,test-condition" "env,bash-environment-variables" "cat,view-file-contents" "tee,write-to-output" "echo,display-content-of-string" "echo_text_>\$file,write-content-to-file" "echo_text_>>\$file,append-content-to-file")
for action in ${actions[@]}; do # for loop is very similar to other language, though [@], semi-colon, do are new
action=${action//-/ } # convert dash to space
action=${action//,/: } # convert comma to colon
action=${action//_text_/ \"sample text\" } # convert _text_ to sample text, note escape character \ to avoid "" having meaning
echo " - ${action//-/ }" >> $file # echo is redirected to file with >>
done
echo ""
echo "File listing and status"
ls -l $file # list file
wc $file # show words
mdless $file # this requires installation, but renders markown from terminal
rm $file # clean up termporary file
File listing and status
-rw-r--r-- 1 johnmortensen wheel 809 Jun 11 10:43 sample.md
15 132 809 sample.md
[0m[0;1;47;90mShow Generated Markdown [0;2;30;47m========================================================[0m
This introductory paragraph and this line and the title above are generated
using tee with the standard input (<<) redirection operator.
[0;1;91m- [0;1;91m[0;97mThis [0;97mbulleted [0;97melement [0;97mis [0;97mstill [0;97mpart [0;97mof [0;97mthe [0;97mtee [0;97mbody.
[0;1;91m- [0;1;91m[0;97mThis [0;97mbulleted [0;97melement [0;97mand [0;97mlines [0;97mbelow [0;97mare [0;97mgenerated [0;97musing [0;97mecho [0;97mwith [0;97mstandard
[0;97moutput [0;97m(>>) [0;97mredirection [0;97moperator.
[0;1;91m- [0;1;91m[0;97mThe [0;97mlist [0;97mdefinition, [0;97mas [0;97mis, [0;97mis [0;97musing [0;97mspace [0;97mto [0;97mseperate [0;97mlines. [0;97mThus [0;97mthe [0;97muse [0;97mof
[0;97mcommas [0;97mand [0;97mhyphens [0;97min [0;97moutput.
[0;1;91m- [0;1;91m[0;97mls: [0;97mlist [0;97mdirectory
[0;1;91m- [0;1;91m[0;97mcd: [0;97mchange [0;97mdirectory
[0;1;91m- [0;1;91m[0;97mpwd: [0;97mpresent [0;97mworking [0;97mdirectory
[0;1;91m- [0;1;91m[0;97mif [0;97mthen [0;97mfi: [0;97mtest [0;97mcondition
[0;1;91m- [0;1;91m[0;97menv: [0;97mbash [0;97menvironment [0;97mvariables
[0;1;91m- [0;1;91m[0;97mcat: [0;97mview [0;97mfile [0;97mcontents
[0;1;91m- [0;1;91m[0;97mtee: [0;97mwrite [0;97mto [0;97moutput
[0;1;91m- [0;1;91m[0;97mecho: [0;97mdisplay [0;97mcontent [0;97mof [0;97mstring
[0;1;91m- [0;1;91m[0;97mecho [0;97m"sample [0;97mtext" [0;97m>$file: [0;97mwrite [0;97mcontent [0;97mto [0;97mfile
[0;1;91m- [0;1;91m[0;97mecho [0;97m"sample [0;97mtext" [0;97m>>$file: [0;97mappend [0;97mcontent [0;97mto [0;97mfile
[0m
Hack Preparation.
Review Tool Setup Procedures and think about some thing you could verify through a Shell notebook.
- Come up with your own student view of this procedure to show your tools are installed.
- Name and create notes on some Linux commands you will use frequently.
- Is there anything we use to verify tools we install? Review versions checks.
- Is there anything we could verify with Anaconda? or WSL?
- How would you update a repository? Could you do that in script above?