Thursday, April 11, 2024

SSIS – Unicode data is odd byte size for column. Should be even byte size

I'm posting this here as I always forget what the fix is when I run into this error. Massive thanks to LSDBTECH for this fix.

When using a script component in SSIS with a Unicode text stream (DT_NTEXT) output column to an NVARCHAR(MAX) database column you can hit the error:

Unicode data is odd byte size for column <column number>. Should be even byte size

This usually happens because the method for adding the large content takes a byte array argument, and the common way to convert the string to a byte array is using Encoding.UTF8.GetBytes function. The blog post linked above has this explanation:

 “Notice that compressed Unicode strings are always an odd number of bytes. This is how SQL Server determines that the string has actually been compressed, because an uncompressed Unicode string—which needs 2 bytes for each character—will always be an even number of bytes” Source

The fix is to use Encoding.Unicode.GetBytes instead. Here's a code snippet, in this case reading an Azure blob

... (Azure connect stuff goes in here)

var content = blob.DownloadText();

BlobPropertiesBuffer.AddRow();
BlobPropertiesBuffer.Name = blob.Name;
BlobPropertiesBuffer.LastModified = blobProperties.LastModified.Value;

// This next line will cause the error "Unicode data is odd byte size for column 3. Should be even byte size"
BlobPropertiesBuffer.Content.AddBlobData(Encoding.UTF8.GetBytes(content));

// Instead use this line
BlobPropertiesBuffer.Content.AddBlobData(Encoding.Unicode.GetBytes(content));

Also noting that adding to the output buffer column for nvarchar(max)/text streams needs to use the function AddBlobData rather than assigning the value to the column which you do for strings and numeric values (see snippet above)

Tuesday, April 9, 2024

Azure Blob Container SAS Key Authentication Error

 Setting up Azure blob container access with a SAS key recently and kept hitting a wall with authentication errors trying to list the container contents. The SAS key had read and list permissions, but even granting full permissions kept coming back with the error

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

I was testing this using PowerShell Get-AzStorageContainer, and this was the problem. Getting the storage container needs a SAS key generated at the storage account level, changing the call to use Get-StorageBlob without specifying a blob name allowed the requests to succeed with the read and list permissions.

Here's the full test code as an example (actual resource names removed)

$storageAccount = "my-storage-account"
$container = "my-container"
$sas = 'sp=rl&st=2024-04-08T04:56:21Z&se=2024-04-08T12:56:21Z&spr=https&sv=2022-11-02&sr=c&sig=sig'

$context = New-AzStorageContext -StorageAccountName $storageAccount -SasToken $sas 

# this line fails with a 403 authentication error
#(Get-AzStorageContainer -Context $context -Name ci-finance-samples -)

# this one will succeed and output a list of blobs in the container
Get-AzStorageBlob -Context $context -Container $container

Azure Data Factory Metadata-Driven Pipelines - 1

 I've had a bit of free time work wise lately so figured I'd finally get onto revamping one of our Azure Data Factory (ADF) extract ...