Accessing Files in a OneDrive Account from Code

The last time, we saw how to authenticate to a Microsoft Live OneDrive account from within a standard Windows forms application. This time, we continue on from that previous post and create routines to make working with the file list much more friendly, followed by showing how to upload and download files from the account.

If you're jumping straight into this without reading the previous article, please be aware this post assumes that you already know how to authenticate to the connected OneDrive account and send rest requests to it. If you don't already know this, I recommend that you at least skim over it so you know how to get an accessToken. The previous article can be helpful.

Getting a File Listing

In the previous post, we added the following code:

private string GetOneDriveRootListing()
{
   var accessToken = GetAccessToken();
   string jsonData;

   string url = string.Format(@"https://apis.live.net/v5.0/me/skydrive?access_token={0}", accessToken);
   using (var client = new WebClient())
   {
      var result = client.OpenRead(new Uri(url));
      var sr = new StreamReader(result);
      jsonData = sr.ReadToEnd();
   }

   OneDriveInfo driveInfo = JsonConvert.DeserializeObject<OneDriveInfo>(jsonData);

   url = string.Format("{0}?access_token{1}", driveInfo.Upload_Location, accessToken);
   using (var client = new WebClient())
   {
      var result = client.OpenRead(new Uri(url));
      var sr = new StreamReader(result);
      jsonData = sr.ReadToEnd();
    }

   return jsonData;
}

This code was used to return the JSON data that represents a complete file listing of the user's root folder.

This is all very useful, but parsing the folder and file list manually is a rather tricky task. File System objects in a OneDrive account can take many different forms; for example, a standard folder might look like this:

      {
         "id": "folder.4515677xxxxxxxxx.4515677xxxxxxxxx!223",
         "from": {
            "id": "4515677xxxxxxxxx"
      },
      "name": "Blog images",
      "description": "",
      "parent_id": "folder.4515677xxxxxxxxx",
      "size": 66529,
      "upload_location":
         "https://apis.live.net/v5.0/folder.4515677xxxxxxxxx.4515677xxxxxxxxx!223/files/",
      "comments_count": 0,
      "comments_enabled": true,
      "is_embeddable": true,
      "count": 3,
      "link":
         "https://onedrive.live.com/redir.aspx?cid=4515677xxxxxxxxx&page=browse&resid=4515677xxxxxxxxx!223&parId=4515677 xxxxxxxxx!161",
      "type": "album",
      "shared_with": {
         "access": "Shared"
      },
      "created_time": "2009-05-23T10:55:58+0000",
      "updated_time": "2010-09-28T18:30:53+0000",
      "client_updated_time": "2010-09-28T18:30:53+0000"
      }

Whereas an image file might look like this:

{
   "id": "file.4515677bdf99b35f.4515677BDF99B35F!1021",
   "from":{
      "name": "Peter Shaw",
      "id": "4515677bdf99b35f"
   },
   "name": "Tiny.png",
   "description": "",
   "parent_id": "folder.4515677bdf99b35f",
   "size": 17630,
   "comments_count": 0,
   "comments_enabled": false,
   "tags_count": 0,
   "tags_enabled": true,
   "is_embeddable": true,
   "picture" "https://public.bn1.livefilestore.com/y2m73vLY1yhhnhqJXy7_bt8Z6_o7u9ypLVhZMz-oD_hLbZs-qicCq_dTKFPjClvc7JH-BNUg-gAnyMMF-SonUfRPDB1LX2yaQ5lBpRg1V6xDi0/Tiny.png.jpg?psid=1",
   "source": "https://public.bn1.livefilestore.com/y2mT99M9gkyFFlj1Q9OxQFNhW0P0Hfji8CIiC60dVdrhPuhfZDQXKSG8QViIxeRmPClWCQIg4z9IR5R2ZlAc8FbZxwrOwsjMEVv_9ZN0cjG3NA/Tiny.png?psid=1",
   "upload_location": "https://apis.live.net/v5.0/file.4515677bdf99b35f.4515677BDF99B35F!1021/content/",
   "images": [
      {
         "height": 117,
         "width": 99,
         "source": "https://public.bn1.livefilestore.com/y2mbp4bsz2-2tbDrVIUC6LYbM1gCneqOvtLoLayTvIU1Aw1iTHH4idHb4w2HOGISHymyob8fz_BP4UAlxlzHH-9BA/Tiny.png.jpg?psid=1&ck=2&ex=720",
         "type": "normal"
      }, {
         "height": 117,
         "width": 99,
         "source": "https://public.bn1.livefilestore.com/y2m_lCuh5snt-qp4ttijSDi7lrr1TrK4zsQEFJi2bMbTm6URGSIiDZeBmSxrMz-6PTNFDTKllNEjKZ5YtknyYj3WQ/Tiny.png.jpg?psid=1&ck=2&ex=720",
         "type": "album"
      }, {
         "height": 96,
         "width": 81,
         "source": "https://public.bn1.livefilestore.com/y2m73vLY1yhhnhqJXy7_bt8Z6_o7u9ypLVhZMz-oD_hLbZs-qicCq_dTKFPjClvc7JH-BNUg-gAnyMMF-SonUfRPDB1LX2yaQ5lBpRg1V6xDi0/Tiny.png.jpg?psid=1",
         "type": "thumbnail"
      }, {
         "height": 117,
         "width": 99,
         "source": "https://public.bn1.livefilestore.com/y2mT99M9gkyFFlj1Q9OxQFNhW0P0Hfji8CIiC60dVdrhPuhfZDQXKSG8QViIxeRmPClWCQIg4z9IR5R2ZlAc8FbZxwrOwsjMEVv_9ZN0cjG3NA/Tiny.png?psid=1",
         "type": "full"
      }
   ],
   "link": "https://onedrive.live.com/redir.aspx?cid=4515677bdf99b35f&page=browse&resid=4515677BDF99B35F!1021&parId=4515677BDF99B35F!161",
   "when_taken": null,
   "height": 117,
   "width": 99,
   "type": "photo",
   "location": null,
   "camera_make": null,
   "camera_model": null,
   "focal_ratio": 0,
   "focal_length": 0,
   "exposure_numerator": 0,
   "exposure_denominator": 0,
   "shared_with": {
      "access": "Just me"
   },
   "created_time": "2014-08-28T11:26:38+0000",
   "updated_time": "2014-08-28T11:46:03+0000",
   "client_updated_time": "2014-08-28T11:46:03+0000"
}

The image file, as you can see straight away, has many extra fields that a folder does not have. It gets even more complicated with movies and Office documents, and also some folders such as "Pictures," which are (as MS refers to them) 'Albums'.

All of this means we can't just use a single simple class object created from the JSON like we did for the 'OneDriveInfo' object as in the code above.

Sidetip: Indecently, if you have Visual Studio 2012 or later, there's a great new feature built into it. Copy your JSON data from whatever source you have, and then in a new class file, rather than just performing a CTRL+V as normal, click Edit, Paste Special in your menus. You'll find 'Paste JSON as Classes', which will take your JSON code and turn in directly into a .NET class ready for you to use. Be careful, though; it can get very complicated, very fast.

If we look at just the basic properties that match on all the objects, we'll see that in actual fact, they are all the same as the whole set of properties in our 'OneDriveInfo' object. This makes sense, because when we get the info for the root of our OneDrive, what we're actually requesting is the file system object for our root folder, and this contains the most basic set of properties we should use.

At this point, we're going to repurpose this object as a base class, so use your Visual Studio refactoring tools (or find and replace) to rename your object class from 'OneDriveInfo' to something more suitable, such as 'FileSystemBase'. We now should be able to deserialize the 'data' property of our file list into a list of 'FileSystemBase' objects, by using newtonsoft.JSON.

Before we do that, however, we need a container to deserialize them into.

Why a Container?

Well, even though the Live API returns a JSON Array of file objects, it wraps that array in a single top-level object. Now, we could if we wanted, simply strip everything before the first '[' and after the last ']' in our data, but because we're using Newtonsoft.JSON, it much easier just to create a class called 'OneDriveFileList' and add the following code:

using System.Collections.Generic;

namespace Onedrive
{
   public class OneDriveFileList
   {
      public List<FileSystemBase> Data { get; set; }

   }
}

This will allow us to take the JSON returned from the file list call:

{
   "data": [
      .... file objects here ....
   ]
}

and save us a whole heap of work.

Our file listing method now simply becomes:

private OneDriveFileList GetOneDriveRootListing()
{
   var accessToken = GetAccessToken();
   string jsonData;

   string url = string.Format(@"https://apis.live.net/v5.0/me/skydrive?access_token={0}"
   using (var client = new WebClient())
   {
      var result =  client.OpenRead(new Uri(url));
      var sr = new StreamReader(result);
      jsonData = sr.ReadToEnd();
   }

   FileSystemBase driveInfo = JsonConvert.DeserializeObject<FileSystemBase>(jsonData);

   url = string.Format("{0}?access_token{1}", driveInfo.Upload_Location, accessToken);
   using (var client = new WebClient())
   {
      var result = client.OpenRead(new Uri(url));
      var sr = new StreamReader(result);
      jsonData = sr.ReadToEnd();
   }

   OneDriveFileList rootList = JsonConvert.DeserializeObject<OneDriveFileList>(jsonData);

   return rootList;
}

Which, when called within the app, should result in the following nicely typed structure:

OneDrive1
Figure 1: Showing the nicely typed structure

Which, like the base object, has an upload location (allowing you to get the file list for that folder) along with the other items such as 'size', 'type', and share permissions.

Once we have the base object, we then can start to extend that class to take account of the other object types. In our demo root folder, for example, we have a small PNG image file. The extra data that this carries is as follows:

"picture": "https://public.bn1.livefilestore.com/y2m73vLY1yhhnhqJXy7_bt8Z6_o7u9ypLVhZMz-oD_hLbZs-qicCq_dTKFPjClvc7JH-BNUg-gAnyMMF-SonUfRPDB1LX2yaQ5lBpRg1V6xDi0/Tiny.png.jpg?psid=1",
"source": "https://public.bn1.livefilestore.com/y2mT99M9gkyFFlj1Q9OxQFNhW0P0Hfji8CIiC60dVdrhPuhfZDQXKSG8QViIxeRmPClWCQIg4z9IR5R2ZlAc8FbZxwrOwsjMEVv_9ZN0cjG3NA/Tiny.png?psid=1",
"images": [
   {
      "height": 117,
      "width": 99,
      "source": "https://public.bn1.livefilestore.com/y2mbp4bsz2-2tbDrVIUC6LYbM1gCneqOvtLoLayTvIU1Aw1iTHH4idHb4w2HOGISHymyob8fz_BP4UAlxlzHH-9BA/Tiny.png.jpg?psid=1&ck=2&ex=720",
      "type": "normal"
   }, {
      "height": 117,
      "width": 99,
      "source": "https://public.bn1.livefilestore.com/y2m_lCuh5snt-qp4ttijSDi7lrr1TrK4zsQEFJi2bMbTm6URGSIiDZeBmSxrMz-6PTNFDTKllNEjKZ5YtknyYj3WQ/Tiny.png.jpg?psid=1&ck=2&ex=720",
      "type": "album"
   }, {
      "height": 96,
      "width": 81,
      "source": "https://public.bn1.livefilestore.com/y2m73vLY1yhhnhqJXy7_bt8Z6_o7u9ypLVhZMz-oD_hLbZs-qicCq_dTKFPjClvc7JH-BNUg-gAnyMMF-SonUfRPDB1LX2yaQ5lBpRg1V6xDi0/Tiny.png.jpg?psid=1",
      "type": "thumbnail"
   }, {
      "height": 117,
      "width": 99,
      "source": "https://public.bn1.livefilestore.com/y2mT99M9gkyFFlj1Q9OxQFNhW0P0Hfji8CIiC60dVdrhPuhfZDQXKSG8QViIxeRmPClWCQIg4z9IR5R2ZlAc8FbZxwrOwsjMEVv_9ZN0cjG3NA/Tiny.png?psid=1",
      "type": "full"
   }
],
"link": "https://onedrive.live.com/redir.aspx?cid=4515677bdf99b35f&page=browse&resid=4515677BDF99B35F!1021&parId=4515677BDF99B35F!161",
"when_taken": null,
"height": 117,
"width": 99,
"type":"photo",
"location": null,
"camera_make": null,
"camera_model": null,
"focal_ratio": 0,
"focal_length": 0,
"exposure_numerator": 0,
"exposure_denominator": 0,

Note that I'm not showing the entire JSON structure here, just the items that are different.

If we turn just those properties into a class, and then inherit that class from file system base, we should end up with a class that looks similar to this:

namespace Onedrive
{

   public class FileSystemImage : FileSystemBase
   {
      public int Tags_Count { get; set; }
      public bool Tags_Enabled { get; set; }
      public string Picture { get; set; }
      public string Source { get; set; }
      public ImageInfo[] Images { get; set; }
      public When_Taken { get; set; }
      public int Height { get; set; }
      public int Width { get; set; }
      public object Location { get; set; }
      public object Camera_Make { get; set; }
      public object Camera_Model { get; set; }
      public int Focal_Ratio { get; set; }
      public int Focal_Length { get; set; }
      public int Exposure_Numerator { get; set; }
      public int Exposure_Denominator { get; set; }
   }

   public class ImageInfo
   {
      public int Height { get; set; }
      public int Width { get; set; }
      public string Source { get; set; }
      public string Type { get; set; }
   }
}

With the addition of these two new classes, and the following method in our Main Form class:

private List<FileSystemImage> GetIamgesFromFileList(OneDriveFileList inputList)
{
   List<FileSystemImage> results = new List<FileSystemImage>();

   using (var client = new WebClient())
   {
      var images = inputList.Data.Where(x => x.Type.Equals("photo"));
      foreach (var fileSystemBase in images)
      {
         string url = string.Format("{0}?access_token={1}", 
            fileSystemBase.Upload_Location.Replace("content/", string.Empty),
            GetAccessToken());
         var webResult = client.OpenRead(new Uri(url));
         var sr = new StreamReader(webResult);
         var jsonData = sr.ReadToEnd();

         results.Add(JsonConvert.DeserializeObject<FileSystemImage>(jsonData));
      }
   }

   return results;
}

It's now trivial to get all the images, and, providing you create types for them, any other file types you wish to handle. All the JSON structures are documented in various pages scattered around the OneDrive developer portal. You may notice in the method for getting image lists that, when I grab the upload location to get the extra JSON information, I do a replace on the string to remove the 'content' leaf from the URL:

fileSystemBase.Upload_Location.Replace("content/", string.Empty)

The upload location for a file actually points to the download link. If you request that location directly, rather than getting a JSON payload as expected, you'll actually get a byte array of the file's contents.

Because all we want is the file information, we strip this off (rather than do things the other way and build the URL), and then just use that as the URL to make the request to. This will, in response, return to us a single JSON object of the same format of the single entry in the original file list, but with all of the properties intact.

There are many other ways we could achieve the same thing. We could, for instance, use a dynamic type and just simply populate the root file list with everything, and then build our lists on the fly from that. This would save the extra network accesses, and if you were using this on a mobile device, it's certainly something you might need to think about. For this post, though, it serves to show how to handle the different file types easily and simply.

All this talk of content leads into the next topic.

Downloading a File

As you've just seen, if you take the 'upload_location' property directly out of the file info object, and the object you're dealing with is a kind of file (rather than a folder or Album), the location URL will have "content/" on the end of it. Making a normal get request against this, as we have been doing with the other requests, will simply and efficiently return you a stream of bytes that consist entirely of the file being requested.

When making a content request, there is no JSON data or any other parts to the payload, just a pure data stream containing the file requested. How you get that file onto disk, or into memory, is entirely up to you. It is, however, sensible to use the StreamReader interface to handle it, especially if the file is something like a video file, which might end up being several hundred megabytes in size.

You can download a file from your OneDrive account by adding the following method to your application:

private void DownloadFile(OneDriveFileList inputList, string fileIdToDownload)
{
   var fileToDownload = inputList.Data.FirstOrDefault(x => x.ID == fileIdToDownload);

   if (fileIdToDownload == null) return;
   using (var client = new WebClient())
   {
      string url = string.Format("{0}?access_token={1}",
         fileToDownload.Upload_Location, GetAccessToken());
      var webResult = client.OpenRead(new Uri(url));
      if (webResult == null) return;
      using (var fileStream = File.Create(fileToDownload.Name))
      {
         webResult.CopyTo(fileStream);
      }
   }
}

Note that you first need to grab a copy of the file list (or at the very least the single object) of the file you want to download. This is because you need the 'upload_location' with its 'content' URL so that you have the location of the files binary data.

In the preceding method, all I've done is passed in the list of our files, and the ID I want to download. The method then uses this to get the file's original file name and upload location, makes the request to the upload location, and then finally streams the bytes into a local file with the same name as the original.

And, that's all there is to it. Downloading files is easy once you have the base file information.

Uploading Files

In the final part of the post, we'll now look at how to upload a file to your OneDrive account.

Like everything we've done so far, this is as simple as using a web client, making a request, processing the JSON that's returned, and acting on it. The major difference from everything we've done so far, however, is that files MUST be sent using either "POST" or "PUT" HTTP verbs.

Of the two, MS actually recommends that, if at all possible, you should try to use "PUT". You can use "POST", but you have to send this by using "multipart/form-data", and because you're not actually sending this from a web page, you'll need to build all the request headers and multipart boundaries, as well as the protocol information by hand.

The easiest way to upload files is with the following code:

private void UploadFileToSkyDrive(string localFile)
{
   string url =
      string.Format(
         @"https://apis.live.net/v5.0/folder.4515677bdf99b35f/
            files/{0}?access_token={1}",
         Path.GetFileName(localFile),
         GetAccessToken());

   using (var client = new WebClient())
   {
      var result = client.UploadData(new Uri(url), "PUT"
      string strResult = Encoding.UTF8.GetString(result);
   }
}

private byte[] FileToByteArray(string fileName)
{
   FileStream fileStream = File.OpenRead(fileName);
   byte[] fileData = new byte[fileStream.Length];
   fileStream.Read(fileData, 0, fileData.Length);
   fileStream.Close();
   return fileData;
}

You'll notice in the URL that I've hard coded the root folder of my OneDrive instance. In reality, you'll most likely want to pass the folder name in as a parameter, allowing you to choose where on your OneDrive you want to upload the file to. Another improvement that you'll also likely want to make is to use a file stream, rather than converting the file to a byte array before sending it. Using a byte array is great for small files, but if you try to use that method for uploading very large files, you'll quickly consume all your available memory.

Once you upload the file, you'll get the obligatory JSON response, which for a successful file should look something like this:

{
   "id": "file.4515677bdf99b35f.4515677BDF99B35F!1022",
   "name": "IMAG0067.jpg",
   "source": "https://public.bn1.livefilestore.com/y2m_tyY82hEKqNplq7jvbmGP6BSf2svpZjM6qQ-8JoxPWS2FzGLahXt-Ue3u35Ptj1oGPsJrwzEY2xlnBxDqTOEinEtmHgaecyPAsPw1Wgo7KQ/IMAG0067.jpg?download&psid=1"
}

You'll see that this contains your file's ID, its original name, and a public link should you want to make it downloadable in a web page.

That's All, Folks?

Except that it really isn't.

The topics we've covered in the last two posts have only just begun to scratch the surface. With the correct scope requests, you can get contact lists from Outlook and profile information. You even can access Office documents and do specialist tasks with those that you can't do on normal files.

The LiveSDK has a massive amount of functionality in it, covering all of Microsoft's Live API. Unfortunately, its sheer size means that it's impractical for me to go any farther than I have done in these posts.

Remember, too, that even though we didn't make any direct use of it, we added the LiveSDK NuGet package to our project too, so it's worth exploring that. Look at the methods and routines in there and study the data structures available.

Finally, remember that this is just a rest-based interface. This means you can use it in PHP, Python, NodeJS, Perl, and just about ANY other language on the planet that's capable of making rest-based requests.

I'll make the sample code/project for these two articles available on my github account at:

http://github.com/shawty

For any one who wishes to clone them, happy OneDriving.....

If you have any suggestions or ideas for posts you'd like to see, please feel free to reach out to me on Twitter as @shawty_ds, or come and hunt me down in the Lidnug Linked.NET users group on Linked-In that I help run. I'll be more than happy to produce posts on specific topics of interest.



Related Articles

Comments

  • not understand

    Posted by al on 11/04/2015 05:51am

    what the code languae and where i write it

    Reply
  • Good Stuff!

    Posted by Gilberto Tezini on 10/17/2015 02:43am

    Very usefull article! Good job!

    Reply
  • Upload gives a 403 Error?

    Posted by Brendon S on 05/28/2015 04:45am

    Hi Peter, very helpful post thanks. In the published article, the FileUpload section of your code - line 12 is incomplete. I took it as: var result = client.UploadData(new Uri(url), "PUT", FileToByteArray(localFile)); I am calling it like: UploadFileToSkyDrive(@"C:\Script.txt"); But I am getting an error "The remote server returned an error: (403) Forbidden." Browsing files and downloading works fine. I looked on the OneDrive developer site but can see no settings that would stop a file upload. The file I am trying to upload is 1Kb of text. Any clues? Regards

    Reply
  • Useless without GetAccessToken

    Posted by dick on 01/03/2015 12:41pm

    Code is completely useless without knowing what GetAccessToken() is.

    • Read Previous Post

      Posted by Keenan on 02/23/2016 09:36am

      Read the previous post to find out about access token and how to get one.

      Reply
    Reply
  • Teaser....

    Posted by Tim on 11/17/2014 01:50pm

    You say in your final closing that it would be better to use a filestream object rather than converting the file to a byte array - any hints on where to start with this? Thanks for the article I have learned a lot! Tim

    • RE: Teaser...

      Posted by Peter Shaw on 12/03/2014 04:45am

      Hi Again Tim: There's a ton of stuff out there that will help you in this respect. I think your referring to using a stream reader however, as I already use a file stream in the demo code above :-) If you are then this stack overflow post will give you a good start: http://stackoverflow.com/questions/12000136/using-statement-filestream-and-or-streamreader-visual-studio-2012-warnings Feel free to find me on twitter and ask me directly, you can usually find me as @shawty_ds

      Reply
    Reply
  • Senior Manager, Programming

    Posted by Richard Thomas on 09/29/2014 03:59am

    Nice article, but the link to the previous article goes to "Creating Webforms with Friendly URLs". I don't see anything about OneDrive authenticating there.

    • RE:

      Posted by Peter Shaw on 09/30/2014 09:08am

      Hi Richard, yes sorry about that, I've informed the site publishers. The article on Authentication was designed to go out before or soon after this one. Hopefully the authentication article will be published soon, and they will link them both together once they do. Regards Shawty

      Reply
    Reply
  • Wrong link?

    Posted by RC on 09/29/2014 02:42am

    Hi, I'm not sure if I'm getting the right link to your previous article but it points to a post on creating WebForms which doesn't tackle anything about authentication or OneDrive at all. I'd love to read the previous post so I could understand this one better. Could you post the right link? :)

    • Authenticating Post

      Posted by Peter Shaw on 11/06/2014 04:59am

      The previous post about authenticating, is now on-line at : http://www.codeguru.com/columns/dotnet/authenticating-a-onedrive-account.html

      Reply
    • RE: Wrong link?

      Posted by Peter Shaw on 09/30/2014 09:09am

      Hi RC, Yes, sorry about that, I've notified the site publishers, hopefully they'll rectify things as soon as possible.

      Reply
    Reply
Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

Most Popular Programming Stories

More for Developers

RSS Feeds

Thanks for your registration, follow us on our social networks to keep up-to-date