Cosmos DB Change Feed Processor NuGet package now available & Working with the change feed support

Cosmos DB Change Feed Processor NuGet package now available

Many database systems have features allowing change data capture or mirroring, for use with live backups, reporting, data warehousing and real time analytics for transactional systems… Azure Cosmos DB has such a feature called the Change Feed API, which was first introduced in May 2017.

The Change Feed API provides a list of new and updated documents in a partition in the order in which the updates were made.

Microsoft has just recently introduced the new Change Feed Processor Library which abstracts the existing Change Feed API to facilitate the distribution of change feed event processing across multiple consumers.

The Change Feed Processor library provides a thread-safe, multiple-process, runtime environment with checkpoint and partition lease management for change feed operations.

The Change Feed Processor Library is available as a NuGet package for .NET development. The library makes actions like these easier to read changes from a change feed across multiple partitions and performing computational actions triggered by the change feed in parallel (aka Complex Event Processing).

Judy Shen from the Microsoft Cosmos DB team has published some sample code on GitHub, demonstrating it’s use.

Working with the change feed support in Azure Cosmos DB

Aravind Ramachandran, Mimi Gentz and Judy Shen also just published an article Working with the change feed support in Azure Cosmos DB on the Azure docs site a few days ago…

2017-7-24

Azure Cosmos DB is a fast and flexible globally replicated database service that is used for storing high-volume transactional and operational data with predictable single-digit millisecond latency for reads and writes. This makes it well-suited for IoT, gaming, retail, and operational logging applications. A common design pattern in these applications is to track changes made to Azure Cosmos DB data, and update materialized views, perform real-time analytics, archive data to cold storage, and trigger notifications on certain events based on these changes. The change feed support in Azure Cosmos DB enables you to build efficient and scalable solutions for each of these patterns.

With change feed support, Azure Cosmos DB provides a sorted list of documents within an Azure Cosmos DB collection in the order in which they were modified. This feed can be used to listen for modifications to data within the collection and perform actions such as:

  • Trigger a call to an API when a document is inserted or modified
  • Perform real-time (stream) processing on updates
  • Synchronize data with a cache, search engine, or data warehouse

Changes in Azure Cosmos DB are persisted and can be processed asynchronously, and distributed across one or more consumers for parallel processing. Let’s look at the APIs for change feed and how you can use them to build scalable real-time applications. This article shows how to work with Azure Cosmos DB change feed and the DocumentDB API.

Azure Cosmos DB Change Feed.png

Note
Change feed support is only provided for the DocumentDB API at this time; the Graph API and Table API are not currently supported.

Use cases and scenarios
Change feed allows for efficient processing of large datasets with a high volume of writes, and offers an alternative to querying entire datasets to identify what has changed. For example, you can perform the following tasks efficiently:

  • Update a cache, search index, or a data warehouse with data stored in Azure Cosmos DB.
  • Implement application-level data tiering and archival, that is, store “hot data” in Azure Cosmos DB, and age out “cold data” to Azure Blob Storage or Azure Data Lake Store.
  • Implement batch analytics on data using Apache Hadoop.
  • Implement lambda pipelines on Azure with Azure Cosmos DB. Azure Cosmos DB provides a scalable database solution that can handle both ingestion and query, and implement lambda architectures with low TCO.
  • Perform zero down-time migrations to another Azure Cosmos DB account with a different partitioning scheme.

Lambda Pipelines with Azure Cosmos DB for ingestion and query

Lambda Pipelines with Azure Cosmos DB for ingestion and query

You can use Azure Cosmos DB to receive and store event data from devices, sensors, infrastructure, and applications, and process these events in real-time with Azure Stream Analytics, Apache Storm, or Apache Spark.

Within web and mobile apps, you can track events such as changes to your customer’s profile, preferences, or location to trigger certain actions like sending push notifications to their devices using Azure Functions or App Services. If you’re using Azure Cosmos DB to build a game, you can, for example, use change feed to implement real-time leaderboards based on scores from completed games.

Read more at https://docs.microsoft.com/en-gb/azure/cosmos-db/change-feed

.NET Conf, a free virtual event for developers

Are you ready to learn all about .NET? .NET Conf 19th to 21st September 2017 is a free virtual conference. #dotnetconf – http://www.dotnetconf.net/


More Details…

The .NET Conf is a free, 3 day virtual developer event co-organized by the .NET community and Microsoft. Some of the speakers lined up so far: –

Scott Hunter

Scott Hunter – Director of Program Management, .NET

Kasey Uhlenhuth

Kasey Uhlenhuth – Program Manager, .NET

Mads Torgersen

Mads Torgersen – C# Language Designer

Mikayla Hutchinson

Mikayla Hutchinson – Principal Program Manager, Xamarin

Scott Hanselman

Scott Hanselman – Principal Program Manager, .NET

What’s in store for you?

“Over the course of the three days you have a wide selection of live sessions that feature speakers from the community and .NET product teams. These are the experts in their field and it is a chance to learn, ask questions live, and get inspired for your next software project.

You will learn to build for web, mobile, desktop, games, services, libraries and more for a variety of platforms and devices all with .NET. We have sessions for everyone, no matter if you are just beginning or are a seasoned engineer. We’ll have presentations on .NET Core and ASP.NET Core, C#, F#, Roslyn, Visual Studio, Xamarin, and much more.”

Check out http://www.dotnetconf.net/ for more details…

Latest SSRS Report Viewer Now Generally Available

Microsoft has just released its latest update to the Report Viewer for SQL Server generally available.

This update replaces the Report Viewer 2015 version and has included several enhancements made for SSRS 2016; including modern browser support, cross-browser printing, report parameter positioning, and a modern look-and-feel.

As with previous releases of the SSRS Report Viewer Control, it is also backwards compatible and works with SSRS versions 2008-2017.

For more details please visit: http://tinyurl.com/ReportViewer2017

To install Microsoft.ReportingServices.ReportViewerControl.WebForms, run the following command in the Package Manager Console

Install-Package Microsoft.ReportingServices.ReportViewerControl.WebForms

See Reporting Services Blog for additional information https://blogs.msdn.microsoft.com/sqlrsteamblog


About the Report Viewer control

We’ve heard from many of you using the existing Report Viewer 2015 control in your existing ASP.NET Web Forms apps and awaiting an updated version with, among other things, modern browser support. This update is for you.

A few things to know about this Report Viewer control:

  • It’s an ASP.NET Web Forms control (there’s a Windows Forms version as well) for your existing apps. (Developing new web apps on ASP.NET Core/MVC and other web frameworks? Know that your scenario is on our radar as well.)
  • It enables you to embed paginated (RDL) reports into your app. (Looking to embed Power BI reports or mobile reports? We’re not building that support into the existing Report Viewer control since it’s limited to ASP.NET Web Forms apps, but you can embed any report using an iframe and the rs:Embed=true URL parameter.)
  • It supersedes the Report Viewer 2015 version and includes several enhancements we made for SSRS 2016: modern browser support, cross-browser printing, report parameter positioning, and a modern look-and-feel.
  • It works with SSRS 2008-2017, and with paginated reports stored in Power BI Report Server.

What’s new in this update

With this update, the Report Viewer control

  • Includes Microsoft.SqlServer.Types and SqlServerSpatial140 assemblies, which you may need to render reports that contain maps.
  • Won’t affect your app’s version of jQuery.
  • Won’t affect your app’s jQuery UI CSS.
  • Supports the SizeToReportContent setting in IE11 and Firefox.
  • Reduces occurrences of redundant scrollbars in IE11.
  • Shows correct toolbar buttons on pages whose ResponseEncoding is not UTF-8.

Install the NuGet package

To install the Report Viewer control into your app,

  1. Open your ASP.NET Web Forms project in Visual Studio 2015 or 2017.
  2. Open the NuGet Package Manager Console (Tools > NuGet Package Manager > Package Manager Console).
  3. Enter this command in the console:
    Install-Package Microsoft.ReportingServices.ReportViewerControl.WebForms

That’s it; your project now has the files you need.

Add a Report Viewer control to your page

If your project doesn’t reference an earlier version of the Report Viewer control, you’re ready to add a Report Viewer control to your page. You’ll need to add

  • A Register tag.
  • A ScriptManager control.
  • The ReportViewer control itself.

Your page will look something like the following:

<%@ Page Language=”C#” AutoEventWireup=”true” CodeFile=”Default.aspx.cs” Inherits=”_Default” %>
<%@ Register Assembly=”Microsoft.ReportViewer.WebForms, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91″ Namespace=”Microsoft.Reporting.WebForms” TagPrefix=”rsweb” %>

<!DOCTYPE html>
<html xmlns=”http://www.w3.org/1999/xhtml”>

<head runat=”server”>
< title></title>
< /head>

<body>
< form id=”form1″ runat=”server”>

  <asp:ScriptManager runat=”server”></asp:ScriptManager>

  <rsweb:ReportViewer ID=”ReportViewer1″ runat=”server” ProcessingMode=”Remote” Width=”850px” Height=”680px”>
<ServerReport ReportServerUrl=”http://your-report-server/reportserver” ReportPath=”/Some Folder/Some Report” />
</rsweb:ReportViewer>

</form>
< /body>

</html>

Just use your ReportServerUrl and ReportPath in place of the sample values above.

Check out this article for a step-by-step walkthrough.

Update an existing app

If your existing project references a previous version of the Report Viewer control, you’ll need to update a few references in your web pages and web.config file; see this article for more info.

Source: https://blogs.msdn.microsoft.com/sqlrsteamblog/2017/07/27/updated-report-viewer-control-now-generally-available/

Creating custom Power BI visuals? (… and adding interactivity to Power BI dashboard)

Power BI VisualIf you are wondering how to create custom visuals for Power BI? Then, handily, there is an increasing number of open source samples and visuals becoming available.

Once such visualisation is the Drilldown Player, release by Microsoft as Open Source, and built in conjunction with their partner Gramener (http://gramener.com).

You can get the code from GitHub @ https://github.com/Microsoft/powerbi-visuals-drilldown-player.

You can get the compiled visual @ https://store.office.com/en-us/app.aspx?assetid=WA104381035&sourcecorrid=bde0be33-be77-400c-a17c-19849a52e1f5&ui=en-US&rs=en-US&ad=US&appredirect=false

Chris Webb recently shared a blog post about using this visual to add interactivity… Creating Animated Reports In Power BI With The Drilldown Player Custom Visual

Chris Webb's BI Blog

Last week I had the chance to do something I have not done before: build a Power BI report to be displayed on a big screen hanging on a wall. To make up for the loss of user interactivity, I used the new Drilldown Player custom visual to cycle through different selections and display a new slice of data every few seconds; Devin Knight’s blog post here has a great summary of how to use it. However I wasn’t happy about the look of the Drilldown Player visual in this particular report: the play/stop/pause buttons aren’t much use if you can’t click on them and the visual doesn’t show all of the values that it is cycling through. As a result I hid the visual behind another one and came up with a different way of displaying the currently-displayed selection.

Here’s a simple example of what I did. Imagine you…

View original post 271 more words

Development Workflows for Data Scientists • Free eBook

Development Workflows for Data Scientists - Free eBook.JPGEnabling Fast, Efficient, and Reproducible Results for Data Science • via GitHub

GitHub partnered with O’Reilly Media to examine how data science and analytics teams at several data-driven organizations are improving the way they define, enforce, and automate development workflows.

Download this complimentary book from: – https://resources.github.com/whitepapers/data-science/ 

Power BI custom visual from Visio

Visualize business process workflows, real-world layouts like factory floor plans, network diagrams, organization structures or any illustration created in Microsoft Visio and easily connect it to Power BI data. Contextually represent Power BI data as colours or text on Visio diagrams. Now drive Operational Intelligence effectively using Visio custom visual.