Updated: Microsoft? More like: My software goes off... Azure AD, Outlook, Office.com, Teams, Authenticator, etc block unlucky folks from logging in

[steven36]

Change backed out to no effect – eek

 

 

Updated Microsoft's online authentication systems are inaccessible for at least some customers today, locking those subscribers out of tons of Redmond-hosted services if they are not already logged in.

 

And, according to the software giant, its attempt to rectify the system failure, by backing out a recent change, did not work.

 

Microsoft, via its cloud platform's status page, said the breakdown started around 2125 UTC (1425 PDT) on September 28. "Customers using Azure Active Directory may experience HTTP 503 errors when accessing the Azure portal," the notification message said, promising further updates as events warrant.

 

The message was later tweaked to:

Starting at approximately 21:25 UTC, a subset of customers in the Azure Public and Azure Government cloud may encounter errors performing authentication operations for a number of Microsoft or Azure services, including access to the Azure Portals. Engineering teams have been engaged and are investigating. The next update will be provided in 60 minutes or as events warrant.

 

The Windows giant also tweeted:

⚠️ We are investigating an issue impacting Azure AD Authentication. More information and updates can be found on the Azure Status page at https://t.co/Dw19fIGsXf

— Azure Support (@AzureSupport) September 28, 2020

Since Microsoft Azure acknowledged the issue, some 2,500 reports of problems accessing MICROS~1's sprawling internet platform, from Azure cloud services to website hosting and virtual machines, have been registered at DownDetector.com. Via Twitter, people have reported that Microsoft's Power BI is inaccessible and complained the Azure status page didn't or still doesn't accurately reflect the status of Azure services.

 

The Azure Status Page, which showed no problems in its grid of services initially, has since been revised to reflect at least problems with Active Directory authentication. In August,

 

Microsoft acknowledged its status page doesn't always contain details about service problems, where "doesn't always" means it misses "more than 95 per cent of our incidents."

Not just Azure cloud platform hit by today's downtime

Beyond Microsoft's public and government cloud wobbles – good luck, JEDI – the authentication system outage has hit its other online services, including Outlook, Office, Teams, and Microsoft Authenticator. If you're not already logged in, it appears, you may be unable to get in and use the cloud-based applications as a result of the ongoing downtime.

 

In a status note on its Microsoft 365 site, the biz said it tried backing out an update in hope of undoing the IT breakdown and found that didn't help. It's still working on the problem:

Title: Can't access Microsoft 365 services

 

User Impact: Users may be unable to access multiple Microsoft 365 services.

 

More info: Users would be unable to access Outlook.com, Microsoft Teams including Teams Live Events, and Office.com. Additionally, Power Platform and Dynamics365 properties are affected by this incident.

 

Existing customer sessions are not impacted and any user who is logged in to an existing session would be able to continue their sessions.

 

Current status: We've identified that reverting the recent change did not alleviate impact to Microsoft services as expected. We're working to explore additional options for mitigation.

 

Scope of impact: Any user may experience access problems for Microsoft 365 services.

 

A Microsoft spokesperson is looking into our inquiry about the outage. We will update this story if there's anything more to tell. ®

Updated to add

 

Source

[Sylence]

I see everything is green at Azure status

where is the outage ?

[mp68terr]

Good! No failure of local applications even if the cloud/net is down.

[steven36]

2 hours ago, Sylence said:

I see everything is green at Azure status

where is the outage ?

You cant go by that you have to go by this

https://status.azure.com/en-us/status/history/

 

Authentication errors across multiple Microsoft or Azure services - Mitigated (Tracking ID SM79-F88)

Summary of Impact: Between approximately 21:25 UTC on Sep 28 2020 and 00:23 UTC on Sep 29 2020, a subset of customers in the Azure Public and Azure Government clouds may have encountered errors performing authentication operations for a number of Microsoft or Azure services, including access to the Azure Portals. Targeted communications will be sent to customers for any residual downstream service impact.

Preliminary Root Cause: A recent configuration change impacted a backend storage layer, which caused latency to authentication requests.

Mitigation: The configuration was rolled back to mitigate the issue.

Next Steps: Services that still experience residual impact will receive separate portal communications. A full Post Incident Report (PIR) will be published within the next 72 hours.

 

The fixed 12 hours ago according to Azure Support

https://mobile.twitter.com/AzureSupport/status/1310781840232333312

 

It was out for over a. day that what all those businesses get for punting all there eggs in one basket.  That means  they lost 24 hours of work because they depend on the cloud.  People  panic like crazy when services  go down like that i bet you they had pitch forks out. They try to push out a update and it caused outages. So they try to roll it back and it did not go right, so it took them a day too get it working again.DEV  calculate it will be out only about 8 hours a  year . But what they think  and what happens  is two different things, no one knows when it comes to the cloud i seen services  i use be out for  a day or 2 like that before. But i have more than one place i back up stuff online so i just switch services .  

[Sylence]

@steven36 this news makes you happy? jeez

 

Cloud is essential nowadays, and these problems happen some times.

 

[steven36]

27 minutes ago, Sylence said:

@steven36 this news makes you happy? jeez

 

 these problems happen some times.

 

 ITs know this  already they expect about 8 hours a year downtime this was over a day . This company is worth almost a  trillion dollars  and a small cloud service could of  fixed it in 24 hours . Why would it make me happy or sad   i dont use it?  I know if i used  it would upset  me i relied on Tech .  Lucky for me  i can live with the cloud or without  it. If something i use goes down  it;s just a inconvenience  for me . I had my internet go off  a week before  and i lived lol.

[Sylence]

bcz of the emote 

[steven36]

25 minutes ago, Sylence said:

bcz of the emote 

Im not  a enterprise  im person  so whats essential for them is not essential for me.  But there  still 6%  of enterprises in the USA  dont even use the Cloud  so it's not essential for all of them  just most of them. 72% use a private cloud  69% have 2 clouds  a public one and a private one they  are the smart ones because they dont put there eggs in just one basket . with  94%  already using it dont no longer  have much  room  to even grow in the USA its about maxed  out . It will have  to grow  somewhere else  were not adopted it  very wide yet, Most  business M$ get is just people switching  from AWS  they already was in the cloud.

[Sylence]

okay lol

[BimBamSmash]

Ouch. My sympathies to all those poor souls who had to clean that mess up. Nothing like an enterprise-wide blackout of any kind to induce prolonged chronic stress.