Tag Archives: HTTP

Gotchas – Microsoft Graph/Outlook REST API throttling, best practices


In recent times, I was working with this customer where they make Graph API calls,

Say,
GET https://graph.microsoft.com/v1.0/users/YYY@XXX.onmicrosoft.com/mailFolders/inbox/messages?$skip=0&$top=30&$select=bodypreview,categories,conversationid,from,hasattachments,id,isdraft,isread,lastmodifieddatetime,parentfolderid,receiveddatetime,replyto,sender,sentdatetime,subject,torecipients
and few other GET (read operations of mail folders/single item handles specifically) and few POST/PATCH calls.

They noticed, once the throttling threshold is exceeded, Microsoft Graph limits any further requests from that client for a period of time.

We looked at the HTTP Status code 429 response. When throttling occurs, Microsoft Graph returns HTTP status code 429 (Too many requests), and the requests fail. A suggested wait time is returned in the response header of the failed request.

{
“error”: {
“code”: “UnknownError”,
“message”: “”,
“innerError”: {
“request-id”: “xxxxxx-esse0-4480-90d4-xxxxxxx”,
“date”: “2018-05-05T09:12:35”
}
}
}

The above is one of the such throttling that I noticed. But throttling behavior can depend on the type and number of requests. In general, Microsoft Graph API is designed to handle a high volume of requests. If an overwhelming number of requests occurs, throttling helps maintain optimal performance and reliability of the Microsoft Graph service. But the throttling limits vary based on the scenario and based on your implementation.

Best practices to handle throttling:
In such scenario’s, I would try one of the following best practices and see if it helps.

  • Reduce the number of operations per request.
  • Reduce the frequency of calls.
  • Avoid immediate retries, because all requests accrue against your usage limits.
  • Reduce the number of GET call for single items.
  • If you do run into throttling, try reduce the number/frequency of calls until no more throttling occurs.
  • When you design your application please pay attention and try to build the ability to handle the clear errors returned to you once you have hit such a limit, and re-ping the endpoint using the suggested time-frames and methods.
  • Being the app developer I would recommend when you implement error handling, use the HTTP error code 429 to detect throttling. The failed response includes the Retry-After field in the response header. Backing off requests using the Retry-After delay is the fastest way to recover from throttling because Microsoft Graph continues to log resource usage while a client is being throttled. So you can try this,
    1. Wait the number of seconds specified in the Retry-After field.
    2. Retry the request.
    3. If the request fails again with a 429 error code, you are still being throttled. Continue to use the recommended Retry-After delay and retry the request until it succeeds.

Related documentation:

For Outlook API & Microsoft Graph, refer Jason’s blogpost https://blogs.msdn.microsoft.com/exchangedev/2017/04/07/throttling-coming-to-outlook-api-and-microsoft-graph/
You can find information on throttling at Microsoft Graph throttling guidance.
For a broader discussion of throttling on the Microsoft Cloud, see Throttling Pattern.
For SharePoint online related ones, please refer https://docs.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online

Hope this helps.

Debug: Collecting HTTP/HTTPS traffic using Fiddler Web Debugger tool


As you aware that the Fiddler tool helps you debug web applications by capturing network traffic between the Internet and test computers. The tool enables you to inspect incoming and outgoing data to monitor and modify requests and responses before the browser receives them. So this post provides the steps for capturing HTTP/HTTPS traffic with Fiddler from your application and saving it as a log file on your local machine. You can use these logs for your troubleshooting purpose.

Here are the steps to collect the traffic:

  • Download Fiddler from here and install it.
  • Run Fiddler and go to Tools -> Fiddler Options.
  • On the HTTPS page, verify that Capture HTTPS Connects is enabled.
  • Verify that Decrypt HTTPS traffic is enabled with the …from all processes option

                      image

 

  • Minimize Fiddler to tray & try repro the issue like,
  • Try replicate the reported issue from your application
  • Make sure that you successfully repro the issue
  • In Fiddler, go to File -> Save -> All Sessions and save it to the local machine/drive.

This will produce a .SAZ file, which you will be able to archive & good to share it for further analysis.

Hope this helps.

Office Developer: Request from Office for Apps going to ajax.aspnetcdn.com getting a 502 error?


In recent times, i was working with this customer where they created an Outlook add-in (Office for Apps/Office.js). They noticed that a request from Office for Apps going to ajax.aspnetcdn.com getting HTTP 502 error. During debugging (network sniffs) we noticed that issue happens that the URLs were blocked by their AV, Proxy Server settings. On fixing them, it worked as expected.

In such scenario, make sure to validate the following:

– Internet Explorer 9 or later must be installed on client computers.

– Please make sure that the affected users/computers must have external Internet access. They must be able to access the following URLs for the respective Office apps to load correctly.

o https://appsforoffice.microsoft.com/lib/x.x/hosted/office.js

o https://ajax.aspnetcdn.com/ajax/x.x/MicrosoftAjax.js

– Any anti-virus solutions, firewalls or network solution is not blocking the Office.js URLs.

–  Make sure that both the URL’s are accessible from IE without any problem – it’s a pre-requisite to start with.

Hope this helps.

Why using Custom Outlook forms and HTTP scenarios not recommended?


In Outlook, custom Outlook forms are not supported for use with HTTP-based mail services (such as Microsoft Hotmail). Outlook does not prevent you from using custom form features if you are using an HTTP-based mail service. However, some features do not work correctly. The features do not work correctly because HTTP-based mail messages are stored in a read-only state on the server. Therefore, we recommend that you do not use custom Outlook forms if you are using an HTTP-based mail service.

While try to log into an Exchange 2003 mailbox by using WebDAV program, you receive a 401 status code in IE


When the user tried to access a Microsoft Exchange Server 2003 mailbox by using a program that uses Web Distributed Authoring and Versioning (WebDAV), Internet Explorer throws 401 HTTP status code. During investigation we found that we may not be able to access the mailbox.

Also we notice that the mailbox has never received an e-mail message, and a user has never logged on to the mailbox. After checking the application log, we notice the following event logged in it:

Event Type: Error
Event Source: MSExchangeIS Mailbox Store
Event Category: Logons
Event ID: 1022
Description:
Logon Failure on database "Storage Group NameMailbox Store (xxxxxxx)"

At end of analysis, we noticed that the mailbox is the recently created one.

By design, when a mailbox object is created in the Exchange information store, the mailbox object is not available by WebDAV until the mailbox is accessed through a client logon. If the mailbox receives an e-mail message, the mailbox object becomes available, but not in a way that permits the WebDAV program access. So, the problem still occurs.

To overcome this, log on to the mailbox by using one of the following clients before you run the WebDAV program:

  • Log on to the mailbox by using a Outlook.
  • Log on to the mailbox by using the Outlook Web Access (OWA).
  • Log on to the mailbox by using a custom application

Happy troubleshooting!

Troubleshooting: IIS/Exchange logs and WebDAV functions


You can collect information about client requests by enabling logging for sites and services. IIS logs are stored in %SystemRoot%system32Logfiles<service_name>. If you examine the IIS/Exchange log file you will see the request for the sample application being handled by IIS – you can notice couple of jargons like GET, PUT, POST, MKCOL, LOCK, UNLOCK, PROPFIND, SEARCH, PROPATCH, SUSBCRIBE, UNSUBSCRIBE, POLL etc.

clip_image002

The three main HTTP functions used are GET, PUT and POST:

  • GET: Retrieves a document 

  • PUT: Puts an item in a folder 

  • POST: Submits an item to a folder

In addition to above, there are few more WebDAV functions that are used:

  • MKCOL: Makes a collection that is in essence the same as a Web Storage System folder. MKCOL allows you to make folders and set their properties. 

  • LOCK: Locks documents to prevent writes. 

  • UNLOCK: Allows writes and creates updates. 

  • PROPFIND: Searches for properties individually or for a whole set of folders. 

  • SEARCH: Provides filter, subquery and sort capabilities to a search. 

  • PROPPATCH: Sets arbitrary properties on an item. 

  • SUBSCRIBE: Adds a user to a list. 

  • UNSUBSCRIBE: Removes a user from a list. 

  • POLL: Checks to see if the notification is fired.

Happy troubleshooting!!

System.Net.Mail: Mail sent via Network delivery throws exception “Command not implemented/supported”


One of the customer reported that whenever he tries to send e-mail by using System.Net.Mail API they get the following error “System.Net.Mail.SmtpException error message: Command not implemented. The server response was: Command not Supported

I had a look at the application. They built a Microsoft Visual Studio 2005 based application and tried to send e-mail by using the System.Net.Mail.SmtpClient class. They use the DeliveryMethod.Network method from the Microsoft .NET Framework 2.0. The application receive the following System.Net.Mail.SmtpException error message: Command not implemented. The server response was: Command not Supported.

During the investigation, we found that this problem occurs because the remote SMTP server does not support the SMTP extended Hello (EHLO) command. The exception occurs when the remote SMTP server returns the 502 reply code and the System.Net.Mail.SmtpClient class expects the 500 reply code.

To overcome this issue, Microsoft released a hot fix downloaded from http://code.msdn.microsoft.com/KB913616/Release/ProjectReleases.aspx?ReleaseId=789. Also you can refer the detailed Knowledge base article regarding this issue and hot fix.

Outlook Issue: Task ‘user@domain.com’ reported error (0x80190193) : ‘The operation failed.’


One of my customer reported a Outlook UI related issue (not Dev issue), as it bugs them having this error every time they  hit the send/receive button. If they try to use Outlook 2010 Beta or Outlook 2007 SP2 generates the error below when doing a full send/receive and their client is connected to Exchange 2007 SP2.

Error: Task ‘user@domain.com’ reported error (0x80190193) : ‘The operation failed.’

When we just looked into this, we found its taking us close to OAB synchronization and it fails – which points to, HTTP Error 403. This problem occurred as they enabled the Secure Sockets Layer (SSL) in the OAB directory in Internet Information Services (IIS) Manager.

Customer tried the following steps, which resolved the issue.

  1. On the server that is running Exchange 2007 Client Access Server (CAS) and that hosts the OAB Web-based distribution point, click Start, click Run, and then type inetmgr.
  2. Expand Web Site, and then expand Default Web Site.
  3. Right-click the OAB virtual directory, and then select Properties.
  4. On the Directory Security tab, under the Secure Communications field, click Edit.
  5. Click to clear the Require secure channel (SSL) check box, and then click OK.
  6. At the command prompt, type iisreset.

FYI: SSL is not required on the OAB virtual directory for domain-connected clients. This is because SSL is not required to download files only. This is the opposite of the default setting for the other Exchange services virtual directories.

Troubleshooting : EWS request throws “The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel” ?


One of my customer updated that whenever they try to make the remote Exchange Web Service (EWS) call from his C#.Net 2008 application (VS.Net 2008 – .Net Framework 3.x), he gets the following error:

The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel

I had a detailed look at their application code.

// Create the Exchange Service Binding        
ExchangeServiceBinding esb = new ExchangeServiceBinding();        
 
// Add its relevant Credentials like user name, password, domain and URL        
esb.Credentials = new NetworkCredential(userName, Password, domain);        
esb.Url = @"https://myexchangeserver/EWS/Exchange.asmx";

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

We checked the credentials passed; it seems everything was fine. But still it was failing whenever we make the request to the server with the above same message. When we checked their environment, we found customer uses the self-signed certificate on the server. This is because, by default, .NET checks whether SSL certificates are signed by a certificate from the Trusted Root Certificate store.

To over-ride this behavior, we need to use the following line in the code, which validate the x509 certificate:

ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true;

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

This will accept all certificates, regardless of why they are invalid, which resolved the customer’s issue.

By validating the X509 certificate provided by the computer running Microsoft Exchange Server 2007 for SSL over HTTP, you help to provide a layer of security for the client application. You must validate certificates before you can start programming with Exchange Web Services proxy classes. If the callback is not set up, the first call will fail with a certificate error.

Happy troubleshooting!!

Behavior changes in Outlook : MAPI & Custom properties


Per the knowledgebase the implementation of MAPI in Outlook has been changed to control how custom properties can be created. To guarantee consistent use of custom properties, custom properties must already be used in the organization or on the Outlook client. As soon as custom properties are being used or are registered, the custom properties can be freely transmitted to other Outlook clients or to servers that are running Exchange Server. The custom properties can also be sent over the Internet.

E-mail messages are typically sent in MIME format over the Internet. When Outlook receives an Internet e-mail message, the message is converted into a MAPI representation. The following are examples of Internet e-mail protocols:

  • POP
  • IMAP
  • HTTP (Hotmail)

By default, Outlook no longer enables Internet mail to create new custom properties. Only properties that are already created in the default mail delivery store are preserved for incoming e-mail messages.

This change mostly affects messages that are sent in encapsulated TNEF (Winmail.dat), where the sender has used the Send using Outlook Rich Text Format option. However, Internet messages that contain X-message header properties are also affected.
Note Messages that contain custom properties that are sent in an Exchange organization are not affected by these changes.

Custom properties can also be saved in .msg files and in .oft files. If a user opens a .msg file that has custom properties, those custom properties are not saved to the default store when the message is saved, forwarded, and so on. Typically, .oft files are used to back up Outlook custom forms. With .oft files, the new behavior applies to all kinds of items. The custom form will not open. Instead, the message will appear in the default form for that particular item type.

In summary, this change in design can cause two things to occur:

  • Outlook ignores non-existing custom properties. If a custom property does not exist in the delivery store, the property will not be created, and its value will be lost. If the custom property already exists in the delivery store, its value is persisted. This change applies to the following:
    • Internet e-mail messages that have TNEF and their embedded messages.
    • S/MIME messages.
    • .msg files when you drop the .msg file into an Outlook item window to add the file to another item. This change also applies to .msg files when you drop the .msg file into the main Outlook window to add the file to a folder or in the Microsoft Word window when you use Word as the e-mail editor.
    • .msg files that a user double-clicks or right-clicks to open.
  • Outlook ignores the one-off form definition. If a one-off form specifies a custom property and that custom property does not exist in the delivery store, the one-off form is not rendered. Instead, the user will see the default form for that particular item type. This change applies to Internet e-mail messages that contain a one-off form definition that is encapsulated in TNEF. This change also applies to .oft files that a user double-clicks or right-clicks to open.

Exchange Server: How to retrieve appointments using C# & WebDAV?


Code Snippet (C#):

   //Declaration part
    string strExchSvrName = “”;
    string strMailbox = “”;
    string strCalendarUri = “”;
    string strDomain = “”;
    string strUserName = “”;
    string strPassword = “”;
    System.Net.HttpWebRequest WebDavRequest = null;
    System.Net.HttpWebResponse WebDavResponse = null;
    System.Net.CredentialCache MyCredentialCache = null;
    byte[] bytes = null;
    System.IO.Stream WebDavRequestStream = null;
 
     // Provide the Exchange server name;
      strExchSvrName = “mydomain.in”;
 
      // Provide Mailbox folder name.
       strMailbox = “Mailbox1”;
 
      //if HTTPS -then provide URI of the user’s calendar folder
      strCalendarUri = “https://” + strExchSvrName + “/exchange/”+ strMailbox + “/Calendar/”;
 
        //if HTTP -then provide URI of the user’s calendar folder
        //strCalendarUri = “http://” + strExchSvrName + “/exchange/” + strMailbox + “/Calendar/”;
 
        //Provide the User name and password of appointment creator. Make sure the user has enough permissions to read
        strUserName = “username”; 
        strDomain = “domain”;
        strPassword = “password”;
 
        //Build the Query
        string query = “<?xml version=”1.0”?><D:searchrequest xmlns:D = “DAV:” >”
                + “<D:sql>SELECT “http://schemas.microsoft.com/exchange/outlookmessageclass”, “DAV:contentclass”, 
“DAV:displayname” FROM “”
+ strCalendarUri + “””
                + “WHERE “DAV:ishidden” = true AND “DAV:isfolder” = false”
                + “</D:sql></D:searchrequest>”;
 
        query = “<?xml version=”1.0″?>”
                + “<dav:searchrequest xmlns:dav=”DAV:”>”
                + “<dav:sql>”
                + “SELECT “
                + “”DAV:displayname”, “ // Appointment Uri portion of the resource
                + “”DAV:href”, “ // Full resource Uri of the appointment
                + “”urn:schemas:httpmail:subject”, “ // Subject of appointment
                + “”urn:schemas:calendar:dtstart”, “ // Start date/time of appointment
                + “”urn:schemas:calendar:dtend”, “ // End date/time of appointment
                + “”urn:schemas:httpmail:textdescription”, “ // Body of appointment
                + “”urn:schemas:calendar:location”, “ // Location of appointment
                + “”urn:schemas:calendar:alldayevent”, “ // Whether appointments an all day appointment
                + “”urn:schemas:calendar:meetingstatus”, “ // Confimed status of the appointment
                + “”urn:schemas:calendar:busystatus”, “ // Comitted status the appointment represents
                + “”http://schemas.microsoft.com/mapi/proptag/x823D0003”, “ // Color of the appointment
                + “”urn:schemas:calendar:reminderoffset”, “ // Reminder offset of the appointment
                + “”DAV:ishidden”, “ // Whether this is hidden
                + “”urn:schemas:calendar:instancetype”, “ // Relation of the appointment to a recurring series
                + “”urn:schemas:calendar:transparent”, “ // Transparency of the appointment to free/busy searches
                + “”urn:schemas:calendar:timezoneid” “ // Display timezone of the appointment
                + “FROM Scope(‘SHALLOW TRAVERSAL OF “” + strCalendarUri + “”‘) “
                + “WHERE “
                + “(“DAV:contentclass” = ‘urn:content-classes:appointment’ ) “ // Item is an appointment
                + “AND (NOT “urn:schemas:calendar:instancetype” = 1) “ // appointment is not the master of a recurring series (Get single
appointments or instances of recurring appointments)
                + “AND (“urn:schemas:calendar:dtend” &gt; ‘” + 
                Convert.ToDateTime(“3-12-2007”).Date.ToUniversalTime().ToString(“yyyy/MM/dd HH:mm:ss”) + “‘)
//Appointment ends after the start of our range
                + “AND (“urn:schemas:calendar:dtstart” &lt; ‘” + 
Convert.ToDateTime(“3-17-2007”).Date.ToUniversalTime().ToString(“yyyy/MM/dd HH:mm:ss”) + “‘)
//Appointment begins before the end of our range
                + “AND (“DAV:displayname” like ‘” + “LS.ImportantDates.” + “%’ ) “
                + “ORDER BY “urn:schemas:calendar:dtstart” “
                + “</dav:sql>”
                + “</dav:searchrequest>”;
 
 
        //Provide credentials required to access the server. Here i used the NTLM
        MyCredentialCache = new System.Net.CredentialCache();
        MyCredentialCache.Add(new System.Uri(strCalendarUri), “NTLM”, new System.Net.NetworkCredential(strUserName, strPassword, strDomain));
 
        //Create the HttpWebRequest object.
        WebDavRequest = (System.Net.HttpWebRequest)HttpWebRequest.Create(strCalendarUri);
 
        // Add the network credentials to the request.
        WebDavRequest.Credentials = MyCredentialCache;
 
        // Specify the PROPPATCH method.
        WebDavRequest.Method = “SEARCH”;
 
        // Encode the body using UTF-8.
        bytes = Encoding.UTF8.GetBytes(query);
 
        // Set the content header length.
        WebDavRequest.ContentLength = bytes.Length;
 
        // Get a reference to the request stream.
        WebDavRequestStream = WebDavRequest.GetRequestStream();
 
        // Write the message body to the request stream.
        WebDavRequestStream.Write(bytes, 0, bytes.Length);
 
        // Release the connection
        WebDavRequestStream.Close();
 
        // Set the content type header.
        WebDavRequest.ContentType = “text/xml”;
 
       WebDavResponse = (System.Net.HttpWebResponse)WebDavRequest.GetResponse();
       Response.Write(“Calendar : “);
  Response.Write(“Status Code: “ + WebDavResponse.StatusCode + “Description: “ + WebDavResponse.StatusDescription);
       using (System.IO.StreamReader streamReader = new System.IO.StreamReader(WebDavResponse.GetResponseStream()))
       {
             Response.Write(streamReader.ReadToEnd());
       }
       WebDavResponse.Close();
        
 
     You can get more related samples and detailed information from Dan’s blog. For more information you can also refer this article .csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }