The sempy Python library in Fabric is quite powerful and offers a
range of capabilities that make working with Fabric environments more
efficient and flexible. One of the most useful features I’ve
implemented in my projects is the ability to trigger semantic model
refreshes directly from a Spark notebook.
This approach allows to trigger the refresh right after the data load is finished.
import sempy.fabric as fabric
import time
workspace = fabric.resolve_workspace_name()
dataset = "My_Amazing_Semantic_Model"
refresh_request_id = fabric.refresh_dataset(workspace=workspace, dataset=dataset, refresh_type="full")
# refresh api is async, so we need to poll until it completes
while True:
time.sleep(60)
res = fabric.get_refresh_execution_details(
dataset=dataset, workspace=workspace, refresh_request_id=refresh_request_id
)
if res.status != "Unknown":
break
if res.status == "Completed":
print(f"Refresh Dataset completed with success {res.extended_status}")
else:
errors = res.messages["Message"].str.cat(sep="\n")
raise Exception(f"Refresh Dataset failed: \n\n{errors}")
This script starts by resolving the current workspace and identifying the target dataset. It then initiates a full refresh of that dataset and waits for the process to complete, checking periodically for updates.
If the refresh fails, it will display the error messages. So you can call this notebook from a Master one and check the error there.
While this example works well for standard use cases, there are some enhancements you might consider depending on your environment:
- Adjusting the polling interval
(time.sleep(60))to better suit your dataset’s refresh duration. - Adding retry logic in case of transient failures.
- Implementing logging or notifications to track refresh success or failure across larger deployments.
- Call this inside a parallel loop to refresh several datasets at once
With these improvements, the process becomes even more robust and suitable for enterprise-level scenarios.
Its also a lot cleaner than bending Data/Fabric Factory to do calls and loops to an API.
Have fun!