Uploading and get files to Azure Blob Storage using the REST API and Postman

https://www.petecodes.co.uk/uploading-files-to-azure-blob-storage-using-the-rest-api-and-postman/

Uploading files to Azure Blob Storage using the REST API and Postman

September 14, 2021 PeteCodes

There are a plethora of different ways to upload files to blob storage. One of the most convenient is to use the HTTP REST API provided.

For this example, I’ll be using Postman to complete the operation, but you could very easily use any other method capable of making HTTP requests.

Prepare Blob Storage Access

The first thing we need to do is to allow access to Postman to be able to upload the file.

We’ll be making us of the Shared Access Signature or SAS Method of authorisation here.

From your Storage Account page in the portal, click the “Shared access signature” menu item;

Azure Portal – Storage – Shared Access Signature Menu Item

From here we can create a new Shared Access Signature Key;

Azure Portal – Storage – Shared Access Signature Page

We want to give Postman enough privileges to be able to and and create blobs within containers.

For “Allowed Services”, deselect “Queue” and “Table” from the options, leaving “Blob” and “File” selected;

Azure Portal – Storage – Shared Access Signature – Allowed Services

For “Allowed Resource Types”, selected just “Object”;

Azure Portal – Storage – Shared Access Signature – Allowed Resource Types

For “Allowed Permissions”, select only “Write”, “Add” and “Create”;

Azure Portal – Storage – Shared Access Signature – Allowed Permissions

We can leave “Blob Versioning Permissions” and “Allowed Blog Index Permissions” at their default of selected for this example.

For “Start and expiry date/time”, I’ve chosen to set this SAS to be valid for one year by setting the expiry to be one year from the start date;

Azure Portal – Storage – Shared Access Signature – Validity

Leave the “IP addresses”, “Allowed Protocols”, “Preferred Routing Tier” and Signing Key at their defaults.

We can now hit the blue “Generate SAS and Connection String” button to generate our authorization values;

Azure Portal – Storage – Shared Access Signature – SAS and Connection Strings

We’ll be copying the “Blob service SAS URL”, so hit the copy button to the right of that box;

Azure Portal – Storage – Shared Access Signature – Blob Service SAS URL

Create a Container

Next we need a container to house our file when we upload it.

Click on the “Containers” menu item on the left;

Azure Portal – Storage – Containers Menu Item

Click the “+ Container” button;

Azure Portal – Storage – Add Container

Give your container a name, I’ve chosen “fileuploads”;

Azure Portal – Storage – Add Container Options

Hit the blue “Create” button to create your container.

Sending a PUT request using Postman

We can now use Postman to upload our file to our new container using the SAS key we’ve generated.

First, create yourself a simple text file somewhere on your hard drive… I created a file called “test1.txt”. Stick some simple text in it… I added “Hello!” to my file.

Windows – Create test file for uploading

Open up Postman and hit the “+ New” button;

Postman “+ New” Button

Choose the “Request” option;

Postman – New Request

Give your request a name, I chose “Upload File to Blob Storage”, and select an existing collection, or click the “+ Create Folder” link and create a new collection. Then hit the “Save to xxx” button to save the request;

Postman – Save Request Dialog

From the “Method” drop down select the “PUT” Method;

Select the “Auth” tab below the “Method” drop down. From here, select “API Key” as the Type, then add a “Key” of “x-ms-blob-type” and a value of “BlockBlob”;

Postman – Authorisation Header

Click the “Body” tab. Select “binary” as the type, which will show us a “Select File” button;

Postman – Binary Body Type

Click the “Select File” button and choose the text file you created earlier. the “Select File” button will be replaced with the name of your file;

Postman – Selected Body File

Next, paste in the “Blob Service SAS URL” value you copied earlier into the “Enter request URL” box;

Postman – Paste “Blob Service SAS URL” into Request URL Box

We now need to add in some information into the “Blob Service SAS URL” value we in order to select the container we’d like to upload our file into, and also give our uploaded file a name.

Our copied value will look something like this (I’ve randomised some values here of course;

https://mystorageaccount.blob.core.windows.net/?sv=2020-08-04&ss=bf&srt=o&sp=wactfx&se=2022-09-14T23:04:32Z&st=2021-09-14T15:04:32Z&spr=https&sig=xhfdhfhdfjkhfiufhuiHKJHrehuierhf%3D

After the “blob.core.windows.net/” section, add in the name of your container (I chose “fileuploads”), and the filename you’d like to give your file (I chose “test1.txt”);

https://mystorageaccount.blob.core.windows.net/fileuploads/test1.txt?sv=2020-08-04&ss=bf&srt=o&sp=wactfx&se=2022-09-14T23:04:32Z&st=2021-09-14T15:04:32Z&spr=https&sig=xhfdhfhdfjkhfiufhuiHKJHrehuierhf%3D

With that, you should now be able to hit the “SEND” button to upload your file, all being well, you should see a “201 Created” response in green at the bottom of the Postman window;

Postman – “201 Created” response

Checking that our file was uploaded

If we head back to the portal and our “Containers” page, we can click on our “fileuploads” container;

Azure Portal – Storage – “fileuploads” container link

We should then see our newly uploaded “test1.txt” file;

Azure Portal – Storage – “test1.txt” file

If we click on that file, we can click the “Edit” tab to view it’s contents;

Azure Portal – Storage – Edit File Tab

And with that, we’ve uploaded a file to blob storage using the REST API and Postman.

====================================================

https://www.postman.com/orange-equinox-450772/azure-event-hub-testing/request/6m40eh5/get-blob

CI/CD for Git pipeline for java

1) download PortableGit-2.46.0-64-bit

2) modify E:\PortableGit-2.46.0-64-bit.7z\etc\gitconfig to remove credentialselector window

[core]

symlinks = false

autocrlf = true

fscache = true

[color]

interactive = true

ui = auto

[help]

format = html

[diff "astextplain"]

textconv = astextplain

[rebase]

autosquash = true

[filter "lfs"]

clean = git-lfs clean -- %f

smudge = git-lfs smudge -- %f

process = git-lfs filter-process

required = true

[credential]

helper = !\"E:/PortableGit-2.46.0-64-bit.7z/mingw64/bin/git-credential-manager.exe\"

3) create package

e:\nuget.exe pack L:\TFS_CATS\FamlinkClassic\sacwis-libraries\Sprint-Active\master\sacwis-web-java\Package.nuspec -OutputDirectory L:\TFS_CATS\FamlinkClassic\sacwis-libraries\Sprint-Active\master\sacwis-web-java\

4)  push package to feed

#e:\nuget.exe push L:\TFS_CATS\FamlinkClassic\sacwis-libraries\Sprint-Active\master\JspcCompile\Jasperlib.1.0.0.nupkg -src #http://dshsutoly3dut06:8080/tfs/CATS/_packaging/test/nuget/v2 -ApiKey DSHSVSTS

5) register package if not registered yet.

$nuget = Get-PackageSource | Where-Object -filterscript {$_.Name -eq 'sawcislib'}

if($nuget -eq $null){Register-PackageSource -Force -Name sawcislib -Location http://dshsutoly3dut06:8080/tfs/CATS/_packaging/test/nuget/v2 -ProviderName NuGet}else{Write-Output 'sawcislib has been registered'}

6) install package

Install-Package -Force -Name sawcislib -Destination $(Build.SourcesDirectory) -Source sawcislib -MinimumVersion 1.0.3 -MaximumVersion 1.0.3

Git in VS code

https://code.visualstudio.com/docs/sourcecontrol/intro-to-git#:~:text=Pick%20an%20existing%20or%20new,init%20on%20the%20command%2Dline.

Introduction to Git in VS Code

Want to easily manage your source code and collaborate with others? Git and GitHub are the tools you need! And with Visual Studio Code, you can set up and use them in a snap. Even if you're a beginner, VS Code's user-friendly interface guides you through common Git actions like pushing and pulling code, creating and merging branches, and committing code changes. And if you're a pro, you'll love the ability to perform Git actions directly within the editor, saving you time and effort compared to using the Git command line. Plus, the seamless workflow between VS Code and Git means you can stay in your editor and get more done.

Set up Git in VS Code

To use Git and GitHub in VS Code, first make sure you have Git installed on your computer. If Git is missing, the Source Control view shows instructions on how to install it. Make sure to restart VS Code afterwards.

Additionally you can sign into VS Code with your GitHub account in the Accounts menu in the lower right of the Activity bar to enable additional features like Settings Sync, but also cloning and publishing repositories from GitHub.

Open a Git repository

VS Code provides several ways to get started in a Git repository, from local to remote cloud-powered environments like GitHub Codespaces.

Clone a repository locally

To clone a repository, run the Git: Clone command in the Command Palette (⇧⌘P), or select the Clone Repository button in the Source Control view.

If you clone from GitHub, VS Code prompts you to authenticate with GitHub. Then, select a repository from the list to clone to your machine. The list contains both public and private repositories.

For other Git providers, enter the repository URL, select Clone, and pick a folder on your local machine to clone the files into. VS Code opens the folder once the repository is cloned on your local machine.

Initialize a repository in a local folder

To initialize a new local Git repository:

1. Pick an existing or new folder on your computer and open it in VS Code.

2. In the Source Control view, select the Initialize Repository button.

   This creates a new Git repository in the current folder, allowing you to start tracking code changes.

   This action is equivalent to running git init on the command-line.

   

Publish local repository to GitHub

You can also initialize and local repository and publish it directly to GitHub. This creates a new repository on your GitHub account, and pushes your local code changes to the remote repository. Having your source code on a remote repository is a great way to back up your code, collaborate with others, and automate your workflow with GitHub Actions.

Use the Publish to GitHub command button in the Source Control view. You can then choose a name and description for the repository, and whether to make it public or private.

Once the repository has been created, VS Code pushes your local code to the remote repository. Your code is now backed up on GitHub, and you can start collaborating with others with commits and pull requests.

Open a GitHub repository in a codespace

GitHub Codespaces lets you open a GitHub repository in a fully configured cloud-based development environment, enabling you to develop in a browser without having to install any software on your local computer. GitHub Codespaces allows free usage for individuals, which makes it easy to get started working on open source projects.

To create a codespace for your GitHub repository:

1. Install the GitHub Codespaces extension in VS Code and sign in with your GitHub account.

2. Run the Codespaces: Create New Codespace command.

3. Select the repository and branch you want to open.

   VS Code opens a new window, which is connected to the codespace. The source code, terminal, and running and debugging are hosted in the remote cloud-based development environment.

   Notice that the File Explorer and Status Bar indicate that the workspace is opened in a codespace.

   

Alternatively, you can also start from a codespace template on the GitHub Codespaces website.

If you already have a codespace open in your browser, run the Codespaces: Open in VS Code Desktopcommand in the browser to connect to the codespace from your local VS Code Desktop.

You can learn more about GitHub Codespaces, including customization such as forwarding ports, in the Developing in a codespace documentation.

Open a GitHub repository remotely

VS Code's remote repository support allows you to browse and edit a GitHub repository without cloning it to your local computer. This is useful for quickly making changes to a remote repository without having to clone the entire codebase to your machine.

1. First install the GitHub Repositories extension.

2. Run the command Remote Repositories: Open Remote Repository... or use the Open Remote Repository button the Explorer view.

3. Search and select the GitHub repository that you want to open.

   

Tip: If you need to execute code or run terminal commands, you can seamlessly switch from a remote repository to a codespace with the command Continue Working on.

Staging and committing code changes

Once you have a Git repository set up, you can start tracking code changes by staging and committingyour newly created and edited code.

Tip: Commit your changes early and often. This makes it easier to revert back to previous versions of your code if needed.

You can access the Source Control view from the Activity Bar to list all changed files in your workspace. You can toggle between a tree view or list view by using the tree/list icon in the Source Control view header.

When you select a file in the Source Control view, the editor shows a diff view that highlights the file changes, compared to the previously committed file.

To stage a file, select the + (plus) icon next to the file in the Source Control view. This adds the file to the Staged Changes section, indicating that it will be included in the next commit.

You can also stage all pending changes at once by selecting the + (plus) icon next to Changes in the Source Control view.

Staged changes can also be discarded by selecting the − (minus) icon next to the file. Similarly, you can discard all staged changes by selecting the − (minus) icon next to Staged Changes in the Source Control view.

To commit your staged changes, type a commit message in the upper text box, and then select the Commit button. This saves your changes to the local Git repository, allowing you to revert to previous versions of your code if needed.

Tip: If you have a GitHub Copilot subscription, and installed the Github Copilot extension, you can use the sparkle icon to generate a commit message.

You can navigate through and review all local file changes and commits in the Timeline view available in the bottom of the Explorer view.

Pushing and pulling remote changes

Once you have made commits to your local Git repository, you can push them to the remote repository. The Sync Changes button indicates how many commits are going to be pushed and pulled. Selecting the Sync Changes button downloads (pull) any new remote commits and uploads (push) new local commits to the remote repository.

Tip: You can enable the Git: Autofetch setting to always get an up-to-date remote commit indicator.

Push and pull can also be performed individually by using their respective commands. You can access these commands from the Source Control menu.

Using branches

In Git, branches allow you to work on multiple versions of your codebase simultaneously. This is useful for experimenting with new features or making large code changes without affecting the main codebase.

The branch indicator in the Status bar shows the current branch and lets you switch to new and existing branches.

To create a new branch, select the branch indicator and choose to create it from the current branch or another local one. Type a name for the new branch, and confirm. VS Code creates a new branch and switches to it, allowing you to make changes to your code without affecting the main branch.

Tip: If you use the GitHub Pull Requests and Issues extension, you can create a branch directly from an issue, which gets you started working in a new local branch and automatically prefills the pull request for you.

After you create a local branch, you can push the branch to the remote repository by selecting Publish Branch in the Source Control view. This creates a new branch on the remote repository, allowing you to collaborate with others in that branch.

Creating and reviewing GitHub pull requests

In Git and GitHub, pull requests (PRs) are a way for collaborators to review and merge code changes from separate branches into the main branch. This enables teams to review and approve code changes before they are incorporated into the main codebase, ensuring that only high-quality changes are merged.

To use pull requests in VS Code, you need to install the GitHub Pull Requests and Issues extension. This extension adds PR and issue tracking functionality to VS Code, allowing you to create, review, and merge PRs from within the editor.

To create a PR, make sure you are on a separate branch from the main branch, and push your code changes to the remote repository. In the Source Control view, select the Create Pull Request button. This opens the PR creation form, where you can enter a title and description for the PR, and choose which branch to merge the changes into. Select Create to create the PR.

To review a PR, select the Review Pull Request button in the Source Control view, and select the PR you want to review. This opens the PR in a new editor window, where you can review the code changes and leave comments. Once you are satisfied with the code changes, you can select the Merge button to merge the PR into the targeted branch.

Learn more about pull requests in VS Code's GitHub documentation.

Using Git in the built-in terminal

As all Git state is kept in the local repository, you can easily switch between VS Code's UI, the built-in terminal, or external tools like GitHub Desktop. You can also set up VS Code as your default Git editor, allowing you to use VS Code to edit commit messages and other Git-related files.

Git Bash on Windows

Git Bash is a popular shell environment for Windows that provides a Unix-like command-line interface for working with Git and other command-line tools. Visual Studio Code's integrated terminal supports Git Bash as a shell, allowing you to seamlessly integrate Git Bash into your development workflow. Installing Git on your Windows machine also installs Git Bash, if it wasn't deselected during the installation steps.

Start by opening View > Terminal (⌃`). Select the dropdown arrow next to the + icon in the terminal panel to pick a new shell to open. If Git Bash is installed, it's shown in the list of terminals and shells. You can toggle between different terminals and shells in the Terminal sidebar. With Git Bash configured in Visual Studio Code, you can now use all of your favorite Git commands directly from the terminal in your code editor.

If you want to set Git Bash as your default shell, open the Terminal dropdown (next to the + icon) and select Select Default Profile. This opens a list of available shells, including Git Bash. Selecting Git Bash sets it as your default shell, and all future terminals will be opened with Git Bash. More advanced terminal tips are available in the terminal documentation.