How to Boost Database Development Productivity on Linux, Docker, and Kubernetes with Microsoft SQL Server 2017
It's that time of year again, the time for cold weather, the holidays, and 'best of' lists. I thought I would come up with a list of the top tools that I use for working with Microsoft Azure. Microsoft ships some great tools for working with Microsoft Azure, especially the new Microsoft Visual Studio tools with version 1.3 of the SDK, but there is always a need for a few helper tools.
I have worked with a lot of tools and frameworks for Microsoft Azure, and these are just a few of the ones I found useful. If you have a favorite tool, please list it in the comment section.
1. Cloud Storage Studio by Cerebrata
Microsoft has shipped a good API for working with Microsoft Azure storage, called the Storage Client Library. But they don't ship any tools that let you work easily with storage as a user. You need an easy to use tool to work with data in blobs, queues and tables. You may have to upload some file to the container, or maybe enter some data into a table. Many developers take time to write their own tools but this breaks my rule: "Only write code that only you can write." In this case you should focus on the application you are writing, and find a good storage management tool.
Cloud Storage Studio is fairly cheap, $50, and is very good. They update it with new features quite often. It works very well with tables and blobs, letting you manage multiple storage accounts. My favorite feature is the ability to drag and drop files into the blob container to upload files. It also make managing the permissions on a container a snap.
2. SQL Azure Migration Wizard
Many companies are migrating traditional applications to the cloud. This usually brings along with it a SQL Server database. Normally when migrating a SQL Server database you are going to migrate it to a SQL Azure Database.
Even with the high degree of compatibility between SQL Azure Database and SQL Server there is a little work to migrating a database. You have to make sure the schema is compatible with the limitations of SQL Azure Database, and you have to find some way to move your data up into the cloud.
Have no fear; SQL Azure Migration Wizard is here to help. It will analyze any SQL Server or SQL Azure database for compatibility issues and fix them for you by generating a new database script. This is based on a large collection of rules that you can customize to suit your own needs.
SQL Azure Migration Wizard can also help you move your data to the cloud. It uses BCP under the covers with special settings to make the transfer as fast as possible, and while keeping it from failing due to a connection timeout.
Microsoft Azure's pricing is consumption based. Simply put, the more you use, the more you pay. Microsoft provides free time to any developer with MSDN. Your receive over $2,700 of free consumption each month, and you can get all the details at http://msdn.microsoft.com/en-us/subscriptions/ee461076.aspx. If you don't have MSDN you can get an introductory account at http://www.microsoft.com/windowsazure/offers/popup/popup.aspx?lang=en&locale=en-US&offer=MS-AZR-0001P. When you have an introductory account you receive some free consumption as well, just not as much as MSDN, your first 25 hours each month are free.
Both of these offers require that you provide a credit card. This is so they can bill you if you go over your free allotment of resources. Their billing portal will send you emails as you approach 100% of your free time, but by time you read and respond to the email it might be too late.
This is where the great and free tool GreyBox comes into play. All you need to do is download GreyBox, install it, and configure it with a management certificate for your Microsoft Azure account and you are ready to go. GreyBox will monitor your account and tell you if you ever leave anything running. The MSDN and Introductory accounts are meant for you to deploy an application, do some testing or do a demo, and then tear it down. It is really easy to forget you left something running and end up with a bill you didn't expect. GreyBox will tell you when you left it running, and help you easily kill it. You can configure how often it checks to see if something is running.
4. Azure Database Upload Utility
Maybe you are migrating an application that has a SQL Server database and you don't want to use SQL Azure Database, but you want to use Microsoft Azure Tables. No problem. Our next top tool helps you with this. The Azure Database Upload Utility will read in your SQL Server database and upload it into a table in Microsoft Azure Storage.
Because Windows Azure Tables do not have relationships with other tables, you can only upload a table, a view, or the results of a query. Perhaps a future version will be able to mash up the data tables into a multi-schema Azure Table.
You don't have to import all of the columns from the source. You can pick which ones you would like to migrate. The tool will also map unsupported data types to a data type that Windows Azure Tables knows. Two things I wish the tool would be updated to support would be to create the table you want to move your data into (it has to be pre-existing today, which is something they call out in the docs) and better support for selecting a smart partition key.
In any case, this will get your data into the cloud, and then you can tweak it from there.
5. Azure Grid
When some people think of cloud computing they really conjure up grid computing in their minds. The concept that a large task (usually mathematical in nature) can magically be ripped to small bits, flung onto a large array of CPU's for crunching, and recollected when done for an answer is very enticing. Cloud computing and grid computing are not the same. But, cloud computing is the dream platform for a grid computing scenario, with access to limitless hardware and other resources, this lets your grid get super big as needed.
Like all developers, when a customer needs a grid solution, the developer usually starts building it from scratch. There are a lot of pieces that need to be built. You have to have a controller that breaks the problem down and manages the work chunks and allocates cloud resources appropriately. You have to have a worker harness that can process the work (local to the CPU it is running on), safely run the supplied code to do the work, and then respond with the results, and you have to develop an API so you can call this grid from the application that is looking for the answer.
You can do all of that. OR, you can download Azure Grid. Azure Grid is from those fine folks over at Neudesic (including David Pallman), based on their more sophisticated Neudesic Grid Computing Framework. I love it when companies contribute to open source. This framework makes it easy to set up a grid, define the code and data to be given to each node, and to control it all from a nice WPF client application. The client app helps you monitor the progress of the entire job with a neat visualization of what is happening. Frankly this is something you just download and play with, even if you don't need grid computing.
6. Lokad Cloud
Lokad Cloud bills itself as a .NET framework object to cloud mapper. This makes me think of nHibernate to tell you the truth, and doesn't really tell you all of what Lokad can really do. It is a broad library that you use from the .NET framework to simplify some of the harder tasks you might want to do with Mirosoft Azure.
When anyone uses a Windows Azure Queue they end up writing some sort of infinite loop to poll the queue, and hand off message processing to some other method. Early in the development of Windows Azure the Storage Client Library had this feature that let you set up a polling process in the background, and have it call into your code when a message arrived. It was removed right before RTM. This feature exists in Lokad, and I love it. Lokad will setup a service to monitor and work with your queue, strongly typing the messages that are sent and received to that queue (which is super helpful).
It provides a task scheduler for Windows Azure (using the queuing system). Many customers ask for this capability. Lokad can strongly type your blob storage too. It can help with working with your logs, and also helps with scaling your application. This is useful as Microsoft Azure doesn't provide any auto-scaling features yet. Using Lokad you can make your application smart enough to scale itself.
7. Windows Azure Service Management CmdLets
Microsoft Azure has a great service management API. It lets you do practically everything you can do on the portal. The problem is that it is REST based, and as a developer you might want a higher level of abstraction to work with when you are trying to automate your deployments and management routines with script.
Enter the great Windows Azure Service Management CmdLets. These are easily downloadable and imported into your PowerShell environment. There are scriptlets for every little task you think you might need, making it easy to script out that upgrade, or the deployment that is part of your continuous integration process. These include what you need to setup the environment, do a complete deploy, and upgrade, and then a removal of your application.
As with any healthy platform, Windows Azure has a great community of tools and helpers to make a developer's life easier. Many of them can cut some serious time off of your project, whether that is by making small tasks easier (working with data in a table) or by providing some serious application building blocks. So take some time off from work, get an introductory account, and try some of these tools. There are many more tools out there, and open source projects are a great way to see how other developers have solved some problems you might be facing.