Monday, August 21, 2017

Azure Active Directory (AAD): Where Is My Data Stored?

Situation
A customer wants to use Azure Active Directory (AAD) but needs to know where the data (like user name, credentials and attributes) is stored. On itself a solid question. However, the answer wasn’t easily found. Or better, quite obscure.

The basics
Before the answer is found (and clarified) one most familiarize him/herself with some Azure ‘slang’. In this posting I limit myself to the ones related to this article.

  • Geo: Abbreviation for geography. At this  moment Azure is to be found in 13 geo’s and two more are announced (France & South Africa).
  • Region: Can be looked upon as one HUGE data center, hosting many Azure services. For instance, there is an Azure region in Amsterdam (Netherlands) and one in Dublin (Ireland)
  • Region Pair: Two directly connected Azure regions, placed within the same geography BUT located greater then 300 miles apart (when possible). An Azure Region Pair offers benefits like data residency (except for AAD…), Azure system update isolation, platform provided replication, physical isolation and region recovery order.

Example of a Geo, with its Azure Regions and Region Pair is Geo Europe. This Geo has two Azure Regions: one in Amsterdam (Netherlands), named West Europe and the other Azure Region located in Dublin (Ireland), named North Europe. Together they make up the Region Pair for Geo Europe.

Azure data storage location by default
By default most Azure services are deployed regionally, enabling the customer to specify the Azure Region where their customer data will be stored. This is the case for VMs, storage and Azure SQL databases.

So when you deploy a set of VMs in the Region West Europe with related storage, that data will be stored in Amsterdam (Netherlands). And yes, and some parts of that data will be replicated to North Europe as well since both Regions are part of the same Region Pair. Reasons for this replication might be of an operational nature and/or of data redundancy options selected by the customer.

This is as expected. However it get’s trickier…

USDS (United States of Data Storage)?
However, there ARE exceptions to the above. In quite a few cases customer data will be stored outside by the customer selected Region (and Region Pair as such).

For instance their are some Azure regional services like Azure RemoteApp, Microsoft Cognitive Services, Preview, beta, or other prerelease services and Azure Security Center which data may be transferred and stored globally by Microsoft. And many times it will end up (in some form) in the USA, or United States of Data Storage…

How about AAD?
AAD isn’t an Azure service offered locally, but is designed to run globally. Any Azure service designed to run globally, it doesn’t allow the customer to specify a certain Region where to store the data related to that same Azure service.

And again, Microsoft isn’t very clear about where that data is exactly stored: ‘…Azure Active Directory, which may store Active Directory data globally…’.

To make it even more confusing the same website states: ‘…This does not apply to Active Directory deployments in the United States (where Active Directory data is stored solely in the United States) and in Europe (where Active Directory data is stored in Europe or the United States)…’

Azure services which operate globally are:

  • Content Delivery Network (CDN);
  • Azure Active Directory (AAD);
  • Azure Multi-Factor Authentication (AMFA);
  • Services that provide global routing functions and do not themselves process or store customer data (Eg: Traffic Manager, Azure DNS).

Still not sure where AAD stores its data…
Because Microsoft is a bit elusive about where EXACTLY AAD data is stored, it’s better to look how AAD is made up technically. Many times the technicians don’t do politics Smile.

The article Understand Azure Active Directory architecture is quite recent and very informative. It tells about primary and secondary replicas used for storing AAD data. And the latter ones make it interesting: ‘…which (the secondary replicas) are at data centers that are physically located across different geographies...’.

Basically it tells me that AAD data is replicated globally. And sure, it will turn up in the USA (USDS) as well. As the matter of a fact, it will turn up in every Region servicing Office 365. Simply because without AAD there is no Office 365 consumption.

And for sure, the same article clarifies it even more with the header Data centers: ‘…Azure AD’s replicas are stored in datacenters located throughout the world…’.

Verdict
When using AAD you know for certain that user data (user names, credentials and meta data for instance) ARE replicated globally.

Do I need to worry?
That depends. Know however, that Microsoft goes to extreme lengths to secure your data. Physical access to their data centers is limited to a subset of highly screened people. On top of it all, Microsoft doesn’t allow governments and agencies to access customer data that easily.

And yes, Microsoft offers the Trusted Cloud. Looking at the sheer amount of certifications and data residency guarantees, you can rest assured that Microsoft does its outmost best to offer the most secure cloud services platform ever built.

Alternatives?
Sure, you can look for alternatives. Like Amazon AWS S3. However, the meta data related to those ‘buckets’, also containing customer data, isn’t guaranteed to stay at a certain location either…

Another approach could be using Azure Geo Azure Germany. Because of VERY strict privacy laws, the exceptions for data storage for regional and global Azure services DON’T apply…

Recommended resources
For a better understanding of this article I recommend to read these resources:



Cross Post: Speeding up OpsMgr Dashboards Based On The SQL Visualization Library

Dirk Brinkmann (Microsoft SCOM PFE, based in Germany) has posted an excellent article all about an easy (and undocumented) way to speed up the SCOM/OpsMgr dashboards bases on the SQL Visualization Library MP.

Go here to read all about it.

Thank you Dirk for sharing!

Largest Microsoft Ebook Giveaway!

Ever wanted to know anything about the latest Microsoft technologies, but were afraid to BUY an ebook because todays technologies are changing too fast? So what you buy today is outdated tomorrow?

Fear no longer! Simply download a FREE Microsoft ebook on the topic you want to know more about it and be done with it. Oh, and because it’s FREE why not download many more Microsoft ebooks?

Want to know more? Hunger for more knowledge? Looking for FREE ebooks, reference guides, Step-By-Step Guides, and other informational resources? Go here and be AMAZED, just like me.

A BIG thanks to Microsoft!

PDF: Overview of Microsoft Azure compliance

When you’re about to use Azure and want to know whether it’s compliant with the regulations your company has to met, I strongly advise you to download the PDF Microsoft Azure Compliance Offerings.

As Microsoft describes: ‘…Azure compliance offerings are based on various types of assurances, including formal certifications, attestations, validations, authorizations, and assessments produced by independent third-party auditing firms, as well as contractual amendments, self-assessments, and customer guidance documents produced by Microsoft. Each offering description in this document provides an up to date scope statement indicating which Azure customer-facing services are in scope for the assessment, as well as links to downloadable resources to assist customers with their own compliance obligations. Azure compliance offerings are grouped into four segments: globally applicable, US government, industry specific, and region/country specific…’

Wednesday, July 26, 2017

Holiday

This blog will be silent for the next few weeks because I am going on holiday, enjoying my family to the fullest.
Image result for national lampoon's european vacation
(Picture from the movie ‘National Lampoon's European Vacation’)

After the holiday ‘I’ll be back’ with quite a few postings, like (but not limited to):

  • The 2 last postings for the series about the future of the System Center stack related to Microsoft’s  ‘Cloud & Mobile First’ strategy;
  • Quite a few postings about Azure (IaaS & management);
  • SCOM updates and the lot.

I wish everybody a nice holiday (if not already enjoying it) and see you all later.

Bye!

Thursday, July 20, 2017

‘Mobile First–Cloud First’ Strategy – How About System Center – 05 – SCSM


Advice to the reader
This posting is part of a series of articles. In order to get a full grasp of it, I strongly advise you to start at the beginning of it.

Other postings in the same series:
01 – Kickoff
02 – SCCM
03 – SCOrch
04 – SCDPM


In the fifth posting of this series I’ll write about how System Center Service Manager (SCSM) relates to Microsoft’s Mobile First – Cloud First strategy. Like SCOrch I think that SCSM isn’t going to make it to the cloud…

Ever heard of Service Desk?
The very start of SCSM was a bumpy ride. Originally it was code-named Service Desk and was tested back in 2006, with the release scheduled somewhere in 2008. The beta release ran on 32-bits(!) version of Windows Server 2003, with IIS 6.0, some .NET Frame work versions (of course), SQL Server 2005 and SharePoint Server 2007 Enterprise.

Service Desk was really a beast. Terrible to install, a disaster to ‘run’ (it was slooooooooooooooow) and filled to the brim with bugs. Totally unworkable. Back then I was part of a test team which put the ‘latest & greatest’ of Microsoft’s products through its paces. The whole team was amazed about the pre-beta level of it. Never ever before we bumped into such crappy software. We even wondered whether we had received the proper beta bits…

So none of us was surprised when Microsoft pulled the plug on it and sent the developers back to their drawing boards. In the beginning of 2008 Microsoft officially announced it was delaying the release until 2010, because the beta release had performance and scalability issues. Duh!

Meanwhile a new name was agreed upon: Service Manager.

2010: Say hello to SCSM 2010
In 2010 the totally rewritten SCSM 2010 was publicly released at MMS, Las Vegas. For sure, the code base for SCSM 2010 was totally new, but somehow the developers had succeeded in bringing back some of the issues which plagued Service Desk: performance and scalability issues… Ouch!

Because SCSM 2010 was really the first version (totally rewritten code remember?) it missed out on a lot of functionality. As a result Microsoft quickly brought out Service Pack 1 for it, somewhere in the end of 2010. For SCSM 2010 SP1 in total 4 cumulative updates were published, alongside a few hotfixes.

From 2012x to 2016 in a nutshell
Sure with every new version (2012, 2012 SP1, 2012 R2 and 2016) the performance and scalability issues were partially addressed, but never they really disappeared. As a result SCSM has a track record for being slow and resource hungry. For SCSM 2016 Microsoft claims that data processing throughout has been increased by 4 times.

None the less, the requirements for SCSM 2016 are still something to be taken seriously. For instance, Microsoft recommends physical hosts, 8-core CPU’s and so on. The number of required systems can run over 10+(!), especially when you want to use Data Warehouse cubes and the lot. Even for enterprises this is quite an investment for just ONE tool.

Also with every new version, additional functionality was added. For instance, SCSM 2016 introduced a HTML based Self Service Portal. Unfortunately, the first version of that portal had some serious issues, most of them addressed in Update Rollup #2.

All in all, the evolution from SCSM 2010 up to SCSM 2016 UR#2 has been quite a bumpy ride with many challenges and issues.

Deep integration
Of course, SCSM offers a lot of good stuff as well. It’s just that SCSM is – IMHO – the component of the SC stack with the most challenges. One of the things I like about SCSM is the out-of-the-box integration with other tools and environments.

SCSM can integrate with AD, other System Center stack components (SCOM, SCCM, SCVMM and SCOrch). And – still in preview – you can use the IT Service Management Connector (ITSMC) in OMS Log Analytics to centrally monitor and manage work items in SCSM. As a result, the underlying CMDB is enriched with tons of additional information for the contained CI’s.

SCSM & Azure
At this moment – besides the earlier mentioned ITSMC in OMS – there are no other Azure Connectors available, made by Microsoft that is. There are some open source efforts, like the Azure Automation SCSM Connector on GitHub. But as far as I know, it isn’t fully functional.

Other companies like Gridpro and Cireson, are offering their solutions. But since these companies do have to earn a living as well, their solutions don’t come for free, adding additional costs to your SCSM investment. Still, some of their solutions resolve SCSM pain points for once and for all. So in many cases these products deserve at least a POC.

But still the Azure integration is limited. On top of it all, Microsoft itself doesn’t offer any Azure based SCSM alternatives. Azure Marketplace offers a few third party Service Management solutions (like Avensoft nService for instance) but none of them Microsoft based.

Of course, you could  install SCSM on Azure VMs, but shouldn’t since it’s a resource savvy product, which would bump up Azure consumption (and thus the monthly bill) BIG time.

No Roadmap?!
Until now Microsoft is pretty unclear about their future investments in SCSM. There is nowhere a roadmap to be found. So no one knows – outside Microsoft that is – what will happen with SCSM in the near future. Will there ever be a new version after SCSM 2016? I don’t know for sure. But the signs are the tell tale sign their won’t be…

ServiceNow
In the last years the online service management solution ServiceNow has seen an enormous push and growth. Not just in numbers but also in products and services.

Basically ServiceNow delivers – among tons of other things – SCSM functionality in the cloud. Fast, and reliable. It just works. Also it integrates with many environments, tools and the lot.

Verdict
SCSM has a troublesome codebase which isn’t easily converted to Azure without (again Smile) a required rewrite. When looking at where SCSM stands today, the reputation it has, I dare to say it’s end of the line for SCSM. No follow up in the cloud, nor a phased migration (like SCDPM or SCCM) to it.

Instead Microsoft is silent about the future of SCSM which on itself says a lot. One doesn’t need to speak in order to get the message across.

Combined with the power of ServiceNow, fully cloud based, it’s time to move on. When you don’t run SCSM now, stay away from it. Because anything you put into that CMDB must be migrated to another Service Management solution sooner or later. Instead it’s better to look for alternatives, using todays technologies to the fullest, like ServiceNow or Avensoft nService. For sure, there are other offerings as well. POC them and when they adhere to your company’s standards, use them.

When already running SCSM, upgrade it to the 2016 version. It has Mainstream Support till 11th of January 2022. Time enough to look out for alternatives, whether on-premise or in the cloud. Because SCSM won’t move to the cloud nor will Microsoft invest heavily in it like it did before it adopted their Mobile First – Cloud First strategy.

So don’t wait until it’s 2022, but move away from SCSM before that year, so you can do things on your own terms and speed, not dictated by an end-of-life date set for an already diminishing System Center stack component.

Coming up next
In the sixth posting of this series I’ll write about SCVMM (System Center Virtual Machine Manager). See you all next time.

Monday, July 17, 2017

Azure Stack and Azure Stack Development Kit Q&A

Since Azure Stack is GA, many questions have come forward. Not only about Azure Stack but also about Azure Stack Development Kit. I’ll do my best to answer most questions and refer to the online resources as well.

01: What’s Azure Stack?
As Microsoft states: ‘Microsoft Azure Stack is a hybrid cloud platform that lets you deliver Azure services from your organization’s datacenter…’. Still it sounds like marketing mumbo jumbo.

Basically it means that with Azure Stack your organization has the same Azure technology on-premise available, deeply integrated with the public Azure. Of course, Azure Stack doesn’t offer the same breadth and depth of services as the public Azure, but still it packs awesome cloud power. It’s to be expected that with future updates Azure Stack will offer more and more public Azure based services and technologies, based on the use cases and demands of existing Azure Stack customers.

And because Azure Stack and the public Azure use the same technologies, the end user experience is fully transparent. The same goes for the administration experience. So basically Azure Stack can be looked upon as an extension of Azure.

So yes, one could look at Azure Stack as a kind of private cloud which can be heavily tied into the public Azure, thus creating a super powered hybrid cloud. But there is more.

02: Does Azure Stack require a permanent connection with public Azure?
No, it doesn’t. You can run Azure Stack either in a Connected scenario or Disconnected scenario. In a Connected scenario Azure Stack has a permanent connection with the public Azure. In a Disconnected scenario, Azure Stack doesn’t have a permanent connection.

Even though the first scenario – Connected – makes the most sense, there are enough valid use cases for the Disconnected scenario as well. Think about area’s with a low internet connection density combined with a far away public Azure region. Or how about hospitals, embassies, military installations and bases? The kind of information kept and processed in places like those are valid use cases for the Disconnected scenario.

03: Why should companies use Azure Stack while public Azure offers more services and is more powerful?
Good question! Suppose you’ve got a production facility which generates HUGE amount of data. That data is processed, and the result sets are used further down the production line. In a public Azure setup it would require an enormous data pipeline to Azure in order to get that data across. And when processed, the result sets have to send back as well. Which is egress traffic = money. On top of it all there is latency since the data travels between the factories and Azure.

With Azure Stack, that data is processed locally (no data traffic costs since it’s local LAN, no WAN) and there is no to very small latency.

Another valid use case is app development. Here public Azure is used for development and Azure Stack is used for production, or vice versa.

Or how about sensitive data which – based on regulations and law – isn’t allowed to live in the public cloud? Now you can keep the data onsite (Azure Stack) and use apps living in the public Azure.

And these are just some of the valid use cases for Azure Stack. There are many more, believe me.

04: Does Azure Stack offer the same services as the public Azure?
No, it doesn’t. Which makes sense when you compare the size of an average Azure region compared to an Azure Stack Smile. However, as stated before, the amount of services offered by Azure Stack will grow in the future, based on customer demand and use-/business cases for Azure Stack.

For now(*) Azure Stack offers these foundational services:

  • Compute;
  • Storage;
  • Networking;
  • Key Vault.

On top of it, Azure Stack offers these PaaS services(*):

  • App Service
  • Azure Functions
  • SQL and MySQL databases

(*: This is per 10th of July 2017. Since Azure Stack is in constant development, changes are that the amount of services offered by Azure Stack will have changed over time. Please check Microsoft for the most recent updates and overview of services offered by Azure Stack.)

05: Can I download Azure Stack and install it on spare hardware I’ve got?
No, you can’t. Because Microsoft invests hard to offer you the same Azure experience (pay as you go, consume with no worries about the hardware and so on) with Azure Stack, they had to lock down the hardware on which Azure Stack runs.

Therefore Azure Stack is delivered as a whole package, hardware and software integrated into one. For now HPE, Dell EMC and Lenovo deliver Azure Stack with their own hardware. Soon other hardware vendors will follow suit.

06: So I can’t test drive it? How do I know whether Azure Stack works for me?
Sure you can test drive Azure Stack, POC it or use it as a developer environment. For this Microsoft has specifically developed Azure Stack Development Kit.

You can download it for free and install it on hardware of your choice. Of course there are some requirements to be met for this hardware, but still it’s up to you what vendor to use.

07: What’s Azure Stack Development Kit? Can I use it for production?
As Microsoft states: ‘…It’s a single-node version of Azure Stack, which you can use to evaluate and learn about Azure Stack. You can also use Azure Stack Development Kit as a developer environment, where you can develop using consistent APIs and tooling…’

As such Azure Stack Development Kit isn’t meant for production. It’s meant for POCs and stuff like that. Go here to learn more about it.

08: Do I need to pay for Azure Stack?
Sure you do. But the prices are lower compared to using the public Azure. Which makes sense because your company pays the hardware and operating costs. Check out this Microsoft Azure Packaging & Pricing Sheet (*) for more information.

(*: Please know this sheet will be updated in the future. As such, just Google for Microsoft Azure Packaging and Pricing Sheet and you’ll find the latest version of it.)

09: Is Azure Stack Development Kit free?
Yes, Azure Stack Development Kit itself is free. However, the moment you connect it to (one of) your Azure subscriptions and start moving on-premise workloads to the public Azure, you will be charged for it.

10: Do have some useful links for me?
Sure, hang on. Here are some useful links, all about Azure Stack and/or Azure Stack Development Kit: