A #dataOp ramblings

Adventures in Business Intelligence

Add multiple scans using the Purview API

As I explained in a previous post about Purview with the official Snowflake connector you need to create a scan per database/schema pair. In the customer where I was working there are a few hundred databases plus their schemas. Doing this by using the UI would mean that we will be wasting a lot of time clicking away and be prone to forget some database/schema pair. I’ve explored the documentation about the Scans - REST API and the rest of this post is just some PowerShell code to automate the creation of scans and their schedules in Purview.

Add user(s) to multiple Power BI workspaces quickly

A few days ago I had to add a new team member to more that 50 workspaces. I started dying inside just thinking of going to each workspace and add it using the portal ui. Here’s a little snippet that saved me a few hundred clicks: Login-PowerBI $workspaces = Get-PowerBIWorkSpace $user = "mySpecial.User@MyDomain.com" foreach($workspace in $workspaces) { Write-Host "Adding User to : " $workspace.Name try { Add-PowerBIWorkspaceUser -Id $workspace.Id -UserPrincipalName $user -AccessRight Member } catch { Write-Host "Message: [$($_.

Snowflake Connector for Purview

In the last few weeks I’ve been working with Purview doing some code to integrate Snowflake entities because there wasn’t an official connector. You can imagine my surprise when I’ve noticed last friday that there’s a new unannounced preview connector that does the work for us. PyApacheAtlas If you already read or watched something about the recommended way to create custom entities on Purview you have heard something about this library.

Deprecating Google Spreadsheets Custom Connector

There’s now an official connector to Google Spreadsheets on Power BI With the November 2021 Power BI release, came the official connector support for Google Spreadsheets from Microsoft. As such there’s no more need for my custom connector. The repository on GitHub is now archived. No more Relationship tag contains incorrect attribute. Line 2, position 86 During the last week I’ve tested it so see if it had the same limitations as mine had and I’m happy to report the XML errors when trying to import data from a Google Sheet with a pivot table don’t occur in the new one.

Advent of Code 2020 - Day 16

Today’s puzzle is called ‘Ticket Translation’. We have a bunch of fields in a language we do not understand and we need to figure out which is which. There’s a set of rules we collected in several ways to a document that will serve as the puzzle input. If you want to follow along please download AdventOfCode2020.pbit and check Day 1 post for instructions. Part 1 Our input file is divided into three sections separated by a blank line:

Advent of Code 2020 - Day 15

Rambunctious Recitation is number game that follows the rules of the Van Eck sequence. Here’s a great video from the Numberphile channel on YouTube about it. And here start our troubles because it seems no one has yet found an efficient algorithm for generating this sequence. We will really need to brute force it. How can we resolve this predicament with the tools available to us? If you want to follow along please download AdventOfCode2020.

Advent of Code 2020 - Day 14

The troubles are far from over, this time it’s the docking program on the ferry that’s not compatible with the port computer system. We need to emulate the port’s software by using a bitmasking system to figure out the correct instructions. Let’s see if we are able to implement this logic with Power BI. If you want to follow along please download AdventOfCode2020.pbit and check Day 1 post for instructions.

Advent of Code 2020 - Day 13

We arrived to the island after the ferry trip and we see ourselves in need of transportation to the nearest airport. Fortunately there’s a shuttle bus service between the sea port and the airport that could take us there. Day 13 is a wonderful mathematical puzzle, following the theme of modular arithmetic of the previous one. Let’s dive into it as there some surprises with the numeric limits on Power BI.

Advent of Code 2020 - Day 12

After the waiting room we are at a ferry navigating to the island. There seems to be a problem with the navigation system, and we volunteered to help. We have a set of instructions and we need to make some sense of them to help the captain circumvent the storm. The solution to the problem will be given by calculating the Manhattan distance (sum of the absolute values of its east/west position and its north/south position).

Advent of Code 2020 - Day 11

Today’s puzzle title is “Seating System” but it’s actually a variation of a well known zero-player game called Conway’s Game of Life. Many of us may recall this game as a popular choice for work assignments in computer science courses. Dr. John Conway, the English mathematician who created this game was last year a victim of COVID-19. If you want to know a little more about him you can read this article that was featured this week on Hacker News.