meshBlog

Open Source Cloud Cost Dashboard in under 10 Minutes

By Jelle den Burger•6. September 2021

Cloud cost dashboard build in google data studio displaying cloud billing information in tables and charts. You can build a cloud cost dashboard like this in under 10 minutes.

Three easy steps to get to your cloud cost dashboard

Everyone loves good dashboards. If done well, cloud cost dashboards can give you all the vital information that you need in a single overview. This is incredibly important with managing your cloud costs.

In this blog post, we will show you how you can build a cloud cost dashboard. See your cloud expenditure at a glance in AWS, Microsoft Azure & (soon to be) Google Cloud. The best part: it is completely free and doesn’t take longer than 10 minutes. This is thanks to our open-source multi-cloud CLI Collie which we recently launched on GitHub. To get to your cloud cost dashboard, we will follow along with these steps:

  • Step 1: Preparing cloud cost data with metadata using tags
  • Step 2: Extracting the data from the clouds
  • Step 3: Building a cloud cost dashboard

At the end of this guide, you will have a cloud cost dashboard that looks something like this.

To follow along with this blog post, you need a license for Google Data Studio. It should be included for free in G-Suite. You could also use another dashboard tool such as Microsoft PowerBI. As long as it supports CSV files as data import.

Step 1: Preparing cloud cost data with metadata using tags

Your cloud cost dashboard is only as good as the data it is built upon. That’s why the first and most important step is to prepare the necessary cost data with the right metadata. Without proper metadata, it is going to be difficult to filter and view information from certain angles. One vital part of building good cost data from the public cloud providers is the use of tags. The more tags you use, the more questions you can answer for yourself or your management:

  • Which team (or department) is spending the most in the cloud?
  • Which cost center is spending the most in the cloud?
  • Which cloud platform has the highest usage?
  • How much are we spending on development stages?
  • How is the expenditure of cloud-native projects vs. lift-and-shift projects?
  • Whom do I need to contact for more information about this project?

Step 2: Extracting the data from the clouds

Once you are happy with the metadata you applied to your projects, it is time to export the cost data. This allows it to be imported into your cloud cost dashboard later on. To make the export as easy as possible, we will use our recently launched Collie CLI. Collie can export all cost data with one single command in a CSV file. We will then prepare this CSV file using Google Sheets so it can be used as a data source for the dashboard.

To do this export with Collie we need to execute the following steps:

  1. Before installing: make sure you have properly set up the CLIs of the cloud platforms.
  2. Install Collie as explained here.
  3. Run the cost export command for a given time interval. Make sure to use whole months as the cost data is on a monthly basis. The following command would work for Q1 & Q2 of 2021:

collie tenant costs --from 2021-01-01 --to 2021-06-30 -o csv > q1_q2_2021_export.csv

  1. You should now have a CSV export with the cost data of your cloud(s). The metadata tags are provided as extra columns, which is important for when we build your cloud cost dashboard.

Next up, we will import the CSV data into a Google Sheets spreadsheet. Follow these steps:

  1. Create a new Google Sheet (hint: navigate to https://sheet.new/).
  2. At the menu at the top, open "File" and click "Import". Navigate to the "Upload" tab in the dialog and upload the CSV file from before in this dialog.
  3. Make sure to untick the checkbox that says "Convert text to numbers, dates, and formulas". Confirm the import by clicking "Import Data".
  4. Open the new spreadsheet by clicking "Open now". Make sure to name this new spreadsheet something that you can remember later.

That’s it! We now have a well-prepared spreadsheet that we will use to power your new cloud cost dashboard.

Step 3: Building your cloud cost dashboard!

Okay, the data is ready! We can start building your cloud cost dashboard now. To make things easier, we have already prepared a template for you. You can find it here. Let’s link it to the data source that we set up before. Follow along the next steps to do so:

  1. Open the dashboard link mentioned above and make a copy of the dashboard. To do so, click the settings icon and click "Make a copy" as shown in the screenshot below.

Google Data Studio screenshot showing how to make a copy.

  1. Data Studio will now ask you to enter a new data source. We will use the Google Sheet from before. To do so, click the dropdown below ‚New Datasource‘ and click ‚Create new data source‘.
  2. Select the connector called ‚Google Spreadsheets‘ and a new window should pop up that allows you to search for Google Sheets in your Google Drive. Try to find the one you created before and confirm the connection by clicking the blue ‚Connect‘ button at the top right.
  3. Before creating this new data source, we need to clean up our data types a tiny bit. In the menu that just opened in front of you, scroll to the ‚from‘ dimension and change its type from ‚Date & Time‘ to ‚Date & Time → Year Month‘.
  4. Confirm the creation of the data source. Click the "Add to Report" button at the top right.
  5. At last, copy over your new report by clicking "Copy report".

That’s it! You’re looking at your new cloud cost dashboard. The cloud cost dashboard template we created offers various features:

  • Viewing total costs on a monthly basis
  • Viewing costs per cloud platform
  • Filtering on dates
  • Viewing costs across various tags, in our case: 
    The owner (who owns this cloud account)
Cost Center (a way of allocating costs to budgets)
Environments (development, production, etc)
Zone (is it a cloud-native or lift-and-shift project)
Departments (which unit owns the project)
  • Viewing tenants (and their metadata) with the highest cost.

There is a good chance that you have different metadata than what we used in the dashboard template, which might break one or more charts. We recommend tweaking your cloud cost dashboard to your needs, but we hope that this template helps you off to a great start 🚀

Get Started with your own Cloud Cost Dashboard!

Now that you have reached the end of this post, you see that it doesn’t have to be rocket science to build a powerful cloud cost dashboard. Leverage the power of our open-source Collie CLI and you have the necessary cost data extracted within minutes. Duplicate our Google Data Studio Dashboard, connect it to the CSV export and you’re done!

What questions are you going to answer with your new cloud cost dashboard? Let us know in the comments below!

What’s next?

Collecting costs isn’t the only thing that Collie can do 😉 Curious what else Collie CLI can do for you? Head over to our meshcloud GitHub page and find out!

OR

Want to move your cloud financials to the next level? Head over to our cost management solution and learn how we can help you!