Pure Storage

Using PowerShell to store S3 objects on Pure Storage FlashBlade

Update 12/5/2018: Pure Storage has modified the returned JSON file. The AccessKey is no longer stored as Id, rather it is stored as Name. I’ve updated the code below to reflect that.

Part of my job involves managing our storage fleet, both block storage and some file level services. I recently had a conversation with some developers about methods in which we were storing files. The current process involved multiple layers consisting of applications, servers, virtualization and storage systems. Not that great for a rapidly growing file system.

I suggested to the team that we investigate the use of our Pure Storage FlashBlade investment. The FlashBlade has native S3-compliant object store support, which would make it an ideal platform for what we were looking to do. To prove out that the FlashBlade could be up for the task, and to give some basic examples, I leveraged the AWS Tools for PowerShell to do some file manipulation on a bucket within our FlashBlade. If you’re looking to start doing some object storage on your FlashBlade check out the cmdlets here. Don’t forget to check out the FlashBlade REST API guide to see how you could write some native API integration and bypass the AWS tools entirely.

Create FlashBlade credentials

You will access the FlashBlade using an Access Key and a Secret Key. You’ll want to protect both of these. Note: These instructions were created prior to the introduction of Shared Accounts in Purity. I’ll update these once we have a FlashBlade on the newer Purity OS.

Navigate to the Storage -> Object Store page on your FlashBlade.
Add a new Access Key. The user name here is for documentation purposes — you won’t use this with the APIs. Download the JSON file or copy out the access and secret keys.

Install the AWS Tools

The AWS Tools can be obtained from the PowerShell Gallery:

Install-Module AWSPowerShell -confirm:$false

Create a credential profile

The AWS tools use a credential profile to store your access and secret key for use. They’re encrypted in your user profile on your system. Check out the AWS Tools for PowerShell documentation for more information.

$JSON = Get-Content -Raw -Path <Path to the JSON file created earlier> | ConvertFrom-JSON
Set-AWSCredential -AccessKey $JSON.Name -SecretKey $JSON.secret_access_key -StoreAs default

Bucket Management

# Create a new S3 Bucket
New-S3Bucket -BucketName <name of the bucket> -EndpointUrl <FlashBlade URL>

# List available buckets
Get-S3Bucket -EndpointUrl <FlashBlade URL>

# Delete a bucket
Remove-S3Bucket -BucketName <name of the bucket> -EndpointUrl <flashblade URL>

Object Management

Creating Objects

#Store native content to an object
$Content = @"
<html>
<head><title>PureStorage FlashBlade Object Storage</title></head>
<body><h1>Welcome to PureStorage FlashBlade Object Storage</h1></body>
</html>
"@

Write-S3Object -BucketName <name of bucket to store object in> -Content $Content Key welcome.html -EndpointUrl <FlashBlade URL>

# Upload an existing file to a bucket
Write-S3Object -BucketName <name of bucket to store object in> -File <path to file> (optional: -key: key name for object in bucket) -EndpointUrl <FlashBlade URL>

# Upload multiple files to a bucket
Get-ChildItem <Path to files to upload> -File | Foreach-Object {
Write-S3Object -BucketName <name of bucket to store object in> -File $_.FullName -Key $_.Name -EndpointUrl <FlashBlade URL>
}

# Upload contents of a folder to a bucket
Write-S3Object -BucketName <name of bucket to store files in> -Folder <path to folder> -EndpointUrl <FlashBlade URL>

Listing Objects

# List all objects in a bucket
$BucketContents = Get-S3Object -BucketName <name of bucket to obtain list of objects from> -EndpointUrl <FlashBlade URL>
$BucketContents | Format-Table
$BucketContents | Out-GridView

Removing Objects

# Removing a single known key object
Remove-S3Object -BucketName <name of the bucket object exists in> -key <key of the object> -EndpointUrl <FlashBlade URL>

Footnote

* The FlashArray now supports running containers/virtual machines via Purity Run. If you’re looking to provide NAS services off a FlashArray, check out the Windows File Server Run application for FlashArray.