Im here, Im Shahrokh Mortazavi. How are you Sir? ampgtampgt Great, thank you. ampgtampgt You are in charge of what? Because youre a big and important person. ampgtampgt Im not sure about that.
But I generally run a few teams like a Python team, for Python tools for Visual Studio, Azure Notebooks, we used to do the Azure SDK for Python, that we handed over for the SDK team, and we also did the art in Visual Studio, and most recently, we are the new home for VS Code for Python. ampgtampgt Wow. ampgtampgt So we hired Don Jayamanne, who was one of the extension authors, and now he works at Marcus a full time on the extension. ampgtampgt All right. Would you say thats data science, or its more than that though? ampgtampgt It is more than that. It covers the whole spectrum of what Python does, which is anything from scripting, to data science, to web, to IOT. Lots of stuff. ampgtampgt Very cool. Its nice to see you at Microsoft loves Python. ampgtampgt At last. ampgtampgt So what do you have to show me today? ampgtampgt So today, I want to talk about Azure Notebooks.
So Jupyter Notebooks, think of them as a repel on steroids. Its got code, markdown, interactive graphics and so on. It was started by Fernando Perez and the Brian Granger.
And basically, weve turned it into a free service. Azure Notebooks, notebooks in general are available as part of Azure Machine Learning Studio as part of HDInsight, and we talked to the big cheeses and said, hey can we just turn this loose as a free service as well as a thank you to the open source in Python and Jupyter communities? ampgtampgt Wow. ampgtampgt And they said, Do it. So now, its a playground for running code, publishing your experiments, teaching, and its free both for compute and storage. ampgtampgt Thats amazing.Thats a lot of free computer. ampgtampgt Yes it is.
Its fantastic. ampgtampgt So you go to notebooks.azure.com, and anybody can get started? ampgtampgt Anybody. Its for anyone to use, most of the use currently has been by academics and people who are trying to learn Python or Data Science, but basically, we see people like Harvard, and Cambridge, and Stanford, and all, putting their courses up there, and teaching and lots of courses on EDICS as well. ampgtampgt What is it about Notebooks and Jupyter Notebooks that make them just so great for education? I mean, it mixes code and prose, doesnt it? ampgtampgt Exactly. If you just want to go straight into an ID, you usually have comments in code.
You dont have much visual cues, and this, imagine Word documents but with code cells as well. So, if you take a typical scientific paper, written in PDF, its not that much different than an old Egyptian hieroglyphics paper, right? Actually, that had more graphics in it than a lot of papers today. So this combines all those elements to really tell a complete story. And if you want to do, for example, reproducible research, you can just point at the Notebook and say, if you dont believe me, go run the Notebook, point it at your own data and check out my claims.
So its fantastic for a lot of scenarios in Academia. ampgtampgt It almost seems like Jupyter Notebooks describe the way that Tim BernersLee wanted the Web to be? ampgtampgt Exactly. ampgtampgt Its an active living academic book that is shareable hypermedia. ampgtampgt Exactly. And its interesting to point out that like for given that its running on Python, and underneath of it is the Anaconda Destro like for Azure and most studio, the front end, and like our system, the Notebook system is runs on Linux, their back end runs on Windows. Nobody knows, nobody cares. ampgtampgt Thats cool. So what kind of people are putting these books together and making them available to Academia? ampgtampgt So, we have people like, here, theres a professor at Cambridge University that has put essentially the Engineering 101 Course that are taught where Newton used to teach. ampgtampgt Oh, wow. ampgtampgt Right there, and people just can go and learn Python, and lots of courses material on Engineering 101.
We have others like OReilly that have put their very famous books like Python Data Science Handbook by Jake VanderPlas. The entire book is written as Jupyter Notebooks and anybody can go in there, and clone it and run it, and Im just clicking on random chapters of the Notebook. And here it is. It happens to be on some. ampgtampgt It looks like a book.
But each piece of code can be run. ampgtampgt Its executable. And you dont even have to log in. You can just look at the HTML captures of the Notebook, and youre good to go.
If you want to go actually, run the code you just log in, you can even edit it, and then tomorrow when you relog back in, all your edits and changes right there as well. ampgtampgt I noticed at the top of just that you pick just a random chapter 500 and 19 clones of just that random chapter. ampgtampgt Exactly. It happened to somebody on Hacker News noticed, put it up there and lots of people descended on it, and started cloning it and using it. ampgtampgt Kind of makes you wonder why every language isnt taught in this way. ampgtampgt Its a good question. Like we see thousands of people from EDICS courses that Eric Camplin has put on Azure Notebooks come in and a lot of them are just learning Python, learning A many people are interested in F actually, so we a.netFsharp support as well.
And so, it hits, it resonates with a lot of people from a teaching perspective. Absolutely. ampgtampgt Do I do all of my work in the browser? Or are there any other ways that I might write this kind of code? ampgtampgt So you can Jupyter runs as a client locally?
This is just happens to be the hosted version. So if you have Notebooks locally, you can upload and you can download them and so on. And then, theres rumors thats an Electronic version of this is coming as well. ampgtampgt Oh well. ampgtampgt Yeah. So, should I just give you a quick overview of Notebooks, or? ampgtampgt Please. ampgtampgt Do you think, okay.
All right. So, if you create a library, essentially think of it as a repo. Youll have a collection of Notebooks and data on whatever you want to put up there.
So here I happened to have a Notebook created. So, in cells can be generally classified as markdown cells or code cells. ampgtampgt Okay. ampgtampgt And when I hit Shift Enter or click this little Run button, this code gets executed, meaning it gets sent to an interpreter of some sort and it sends the results back. ampgtampgt Okay. ampgtampgt So, I happen to have a markdown to show these cool Channel nine, logo. You can do math equations using lowtech, and Im going to bother to run these one by one, Im going to do Cell Run All Below.
So anything that was below my current position is going to get executed. ampgtampgt Do some create prerequisites that other cells below it depend on? ampgtampgt Exactly. You can have a sequence, you end up going up and down in a random sequence and thats why Ill talk about Jupyter lab which is the follow on to Notebooks that gives you a little bit more random access. So here, you can get beautiful graphs in whatever discipline you might be in.
You can also do escapes into shelves.
So here, you see bang who am I are print working directory, this gets sent to the whatever show is, if you happen to be on Windows, it gets to command.exe or PowerShell whatever youve set up as your shell, or you can run it here on Linux. Theres a bunch of magic commands that you can essentially again, theyre interpreted specially, and you can get their responses, heres a collection of them. If you do, you can do execute an entire cell by a interpreter, and finally, you can use cURL to download data frames, and then do some data science and so on. ampgtampgt Another cool thing that people use, you were talking about what scenarios people are interested in, we see people turn blogs into complete executable notebooks.
So this is a blog that the Cosmos team did. And it basically points at a readonly database, and you can essentially check out all the key features of Cosmos directly in a notebook without installing anything, and thats the beauty of it. Its zero friction to actually get something and learn from it. ampgtampgt Its amazing.
Again, it gets to my point of why are we doing this in 2017? Why have we not always had an executable living web? ampgtampgt Exactly, I would almost encourage anybody at Microsoft or other places that are writing blogs that have code embedded in them to consider using Azure Notebooks and just putting them up there and let thousands of people use them, run them, and learn about their product whether its Microsoft or somebody elses product. And then, well do automatically all the machine management, resource management, keeping the machines patched, and scaling as thousands of people come in. ampgtampgt This is a slightly orthogonal question, but how do you keep people from abusing this?
If I post this on my blog, what if someone decides to mine Bitcoin with Azure Notebooks? ampgtampgt Right. So, we do keep an eye on things. We make sure there are guards in place to avoid DDoSing. If people did a little bit of Bitcoin mining and so on, we will eventually notice and block them.
But generally, so far, there hasnt been much abuse to the point. ampgtampgt So the occasional forloop might go wild but for the most part, its handled. ampgtampgt Its been okay. Another resource to look at is nbviewer.org. This is a collection, sort of a repo for thousands and thousands, lets call it millions at this point. There are two million notebooks available just on GitHub alone.
And this doesnt count other place. Just like a wealth of high quality notebooks that you can peruse, download, and immediately run it on Azure Notebook. ampgtampgt Just to be clear, theres the notebook file that you can share. And then theres the notebook when it has a URL on Azure Notebooks, or other online Jupyter Notebook, right? So, youre sharing the file. ampgtampgt Yes. If youre sharing it through Azure Notebooks, theres a URL like, for example, for this discussion today is notebooksazure.comsmortaz, which is my log in, libraries, which is all my collection libraries, Channel9 Chat. ampgtampgt And then within that, there is ipynb. ampgtampgt Yes, ipynb is essentially the notebook format file and you can share either the notebook directly or you can share the higher level library that contains all of them. ampgtampgt So then if I go to nbviewer, I can take those notebooks, I could copy them into my own personal library, I could clone them. ampgtampgt Correct. ampgtampgt Share them however I like.
Then take those text files and give them a URL. ampgtampgt Exactly. You can have local notebooks that youve uploaded. You can download them, you can share them. You can have private libraries and public libraries. Another OReilly book that I should mention is Wes McKinneys Python Pydata book.
He is the author of pandas which is the standard package you use for manipulating data frames and its very, very popular. And last but not least, Id like to mention that something that might be of interest to you is, so what I just clicked on, it just started a docker container somewhere in Azure on a Linux VM and put me in there. One of the cool features of Jupyter Notebooks, for people that give a lot of talks, just like I use PowerPoint which is cool, but its mostly static. Now imagine if you had PowerPoint and you could still use like the right, left, up and down and so on to navigate through. But imagine your PowerPoint executed code. ampgtampgt Thats pretty cool.
And RISE is the library? Look at that. Thats crazy. ampgtampgt So, for somebody that gives a lot of talks or webinars and so on which is, by the way, another very, very popular scenario is, we see a lot of local groups or people that give talks at conferences.
Suddenly, like a thousand people or hundreds of people come in and theyre listening to it. Generally, you spend half an hour just setting up your 200 students and says, Oh, this didnt install. That didnt install. All of that goes away. ampgtampgt People could use an iPad or a Chromebook. ampgtampgt Exactly.
Just give them the URL, they click on it, they say clone, they have their own copy of your library, and youre good to go. ampgtampgt Thats fantastic. ampgtampgt Lets see. Do I have any? Lets see it. And then, some of the things, like some of the more advanced features of notebooks that I was going to get to, lets just do a quick run all for the sake of time, is essentially, you can have interactive widgets. ampgtampgt I see. So, you have a function that you create and then you make the input have a UI. ampgtampgt Exactly, and for a little bit more advanced version of them is the networks library that I can basically select different algorithms from other networks.
Its manipulated. And oh, look, whos this guy? So, embedding web pages and articles.
For tutorials and webinars and classes given by professors, essentially, its become the ideal canvas. And last but not least, Id like to point out the next version of Jupyter Notebooks. Its called Jupyter Lab.
So, Jupyter Lab itself. ampgtampgt This looks like an IDE version of Jupyter. ampgtampgt Exactly. After a few years of using Jupyter Notebooks alone, which essentially is just this pane over here, people came to the conclusion, Look, you need a file manager much like VS code and open folder on Visual Studio. And then you need to be able to essentially manipulate Windows and move things around and play around with them.
It also has a text editor. Its got a launcher that I can go and have a terminal that I can do all my machinery management. The good thing here is, the editor that you can actually do like VS code style writing is going to become Monaco. ampgtampgt The code name for the editor that powers VS code itself. ampgtampgt Exactly. The rich editor, the VS code team, also the editor that theyve built, will become the editor here.
And one thing that you probably have noticed by now is like, I didnt show you any debugging. Jupyter environment is completely missing the debugger. The good thing is our team has committed to actually providing the debugger API, the protocol multilingual, and were going to do the first implementation for the Python version and that other people can clone that and use it for their own languages. ampgtampgt Very cool.
Well, that is just a taste of the power and the cool stuff that you can do with Azure Notebooks. Thanks so much for sharing. ampgtampgt My pleasure. ampgtampgt You have got to go and check out Azure Notebooks, and all the great things that are happening around Python that we learned about here on Azure Friday.p