Developing on Staxmanade

Oops - how a simple bit of automation put NuGet services on edge...

(Comments)

This past week I received an email from Microsoft's NuGet team asking if I could look into a bit of an issue with DefinitelyTyped's NuGet package publishing.

Some Background

A really long time ago, I wanted to access DefinitelyTyped packages within Visual Studio via the NuGet package manager. So I quickly wrote up a powershell script to accomplish this. This script has run almost continuously ever since, and primarily without issue.

There's been a couple tweaks/issues along the way - as to be expected, but it's been primarily hands-off.

As of today, these NuGet packages have been downloaded over 5,268,852 times - wow.

What does the automation do?

All of the NuGet packages generated for DefinitelyTyped are run through a build process on the good servers at AppVeyor (Thanks AppVeyor).

Every 2 hours the task does some git-fu to figure out what DT packages have updated (since the last run) and publishes updated NuGet packages for each updated DT package.

The initial problem report:

First let me say that thanks to Yishai and Maarten from Microsoft who brought the issue to my attention and were extremely polite and patient with the raised issue. So thank you, thank you, thank you for the support and being so friendly while working through this...

service status image of problem with nuget

Looking at status.nuget.org

It was pretty easy to see that every 2 hours a large spike in uploads to NuGet was happening.

service status image of problem with nuget

service status image of problem with nuget

While I can't say for certain this incident report on the status page was due to the NuGet automation, it was around the same time the automation was pushing extra builds (and right before I was contacted by Microsoft).

Was that my automation oops?

I didn't recall getting an error email from AppVeyor so I was initially suspicious. But logging in and looking at build history: hmmm. Looking back at my email, looks like I did receive the first failed build email - but must have been busy day as I didn't happen to notice that one email (when I usually do from other projects).

service status image of problem with nuget

YIKES!... so I quickly responded to Microsoft saying I'd shut down the automated portion and dig into it.

The Problem & Resolution

Turns out the problem was due to a large pull request that updated about every package in the DT repo. This meant it had to publish every single package, but for some reason (not showing in the logs) the AppVeyor job was failing at the end and not saving the fact that the packages were being updated on NuGet...

I have a way to run the NuGet publish manually on a local machine so I pulled down the project and ran the complete build. This took quite a while (almost 3 hours) and eventually I discovered the problem.

At the end of the script is a git commit -m {msg} command. This is an important step as it records what has been updated/published. The problem was due to the large number of packages published, the {msg} was so big that it was throwing an error saying the command line command was too long to execute. Causing the system to not complete the cycle and end up re-publishing all nuget packages every 2 hours.

I was able to manually commit with a shorter message and it brought the system back to normal.

And below is NuGet status after the fix.

service status image of problem with nuget

Thanks NuGet!

Turns out the NuGet team put some time into optimizing the publishing process of their service - so maybe there was a benefit to this whole fiasco, but hopefully we won't be hammering the system going forward :)

So I'd like to say thanks again to the NuGet team for you're kind support email and professional way of handling the issue. This is a great example of how Microsoft is helping the OSS community and their support is really taking off and showing promise!

Also Good Point

service status image of problem with nuget

Happy NuGetting!

Hello World with TypeScript and JSX

(Comments)

If you're looking for a solid TypeScript JSX tutorial this is a great resource.

Last night I wanted to play with TypeScripts new support for JSX. In this post I'll walk through my process, and what I learned along the way. Hopefully you find this useful.

While it doesn't yet exist in the current version (at the time of this writing TypeScript is at 1.5), you can however grab a copy of the nightly builds from npm.

Get the most recent nightly build.

npm install -g typescript@next

The rest of this post was run against nightly build Version 1.6.0-dev.20150814.

Given this sample React/JSX

var HelloMessage = React.createClass({
  render: function() {
    return <div>Hello {this.props.name}</div>;
  }
});

React.render(<HelloMessage name="John" />, mountNode);

Create a TypeScript version of a JSX file.

Just like how TypeScript doesn't read .js but looks for .ts files (unless you hack it). TypeScript doesn't read .jsx files. It instead looks for .tsx files.

So if you save the above sample as a helloWorld.tsx, we can then run the tsc compiler against our helloWorld.tsx file.

If I run tsc helloWorld.tsx I get the following errors:

> tsc helloWorld.tsx
helloWorld.tsx(1,20): error TS2304: Cannot find name 'React'.
helloWorld.tsx(3,12): error TS17004: Cannot use JSX unless the '--jsx' flag is provided.
helloWorld.tsx(7,1): error TS2304: Cannot find name 'React'.
helloWorld.tsx(7,14): error TS17004: Cannot use JSX unless the '--jsx' flag is provided.
helloWorld.tsx(7,44): error TS2304: Cannot find name 'mountNode'.

Working through the errors...

I could just show you the final output that compiles, but want to include my learning process (stumbling) as I fumble through and figure out the new command.

Fixing error TS2304: Cannot find name 'React'.

If you've been using TypeScript for any amount of time, the first error should be easy to see. The compiler knows nothing about this thing called React. And I haven't used React with TypeScript before. I don't want to go write a bunch of TypeScript type definitions for react and can easily pull down ones created already by using tsd to install the Definitely Typed definitions for React.

What is tsd?

If you haven't seen TSD before it's a great package manger utility for TypeScript Type Definitions.

It can be easily installed with npm install -g tsd.

Installing React Type Definitions

UPDATE: Originally below I used tsd to install react but if you check out the comments react-global works out better and you can avoid some of the hacks I put in place to compile React below.

tsd install react which tsd will download from Definitely Typed the react.d.ts and place it in ./typings/react/react.d.ts.

I then reference the file in our helloWorld.tsx which gives me the following:

/// <reference path="./typings/react/react.d.ts" />

var HelloMessage = React.createClass({
  render: function() {
    return <div>Hello {this.props.name}</div>;
  }
});

React.render(<HelloMessage name="John" />, mountNode);

When I re-run tsc helloWorld.tsx hoping to get rid of the first error: hmmm

> tsc helloWorld.tsx
helloWorld.tsx(3,20): error TS2304: Cannot find name 'React'.
helloWorld.tsx(5,12): error TS17004: Cannot use JSX unless the '--jsx' flag is provided.
helloWorld.tsx(9,1): error TS2304: Cannot find name 'React'.
helloWorld.tsx(9,14): error TS17004: Cannot use JSX unless the '--jsx' flag is provided.
helloWorld.tsx(9,44): error TS2304: Cannot find name 'mountNode'.

Well, that didn't get rid of our error TS2304: Cannot find name 'React'.. This threw me for a bit but eventually figured out that you need set it up by adding import React = __React;.

So that gives us this:

/// <reference path="./typings/react/react.d.ts" />
import React = __React;

var HelloMessage = React.createClass({
  render: function() {
    return <div>Hello {this.props.name}</div>;
  }
});

React.render(<HelloMessage name="John" />, mountNode);

Now we should see some errors going away. And we do...

> tsc helloWorld.tsx
helloWorld.tsx(6,12): error TS17004: Cannot use JSX unless the '--jsx' flag is provided.
helloWorld.tsx(10,14): error TS2607: JSX element class does not support attributes because it does not have a 'props' property
helloWorld.tsx(10,14): error TS17004: Cannot use JSX unless the '--jsx' flag is provided.
helloWorld.tsx(10,44): error TS2304: Cannot find name 'mountNode'.

Fixing error TS17004: Cannot use JSX unless the '--jsx' flag is provided.

The next error is new to me, but it makes some sense, so I add the --jsx flag to tsc and try tsc --jsx helloWorld.tsx, but looks like I missed a parameter to --jsx.

> tsc --jsx helloWorld.tsx
message TS6081: Argument for '--jsx' must be 'preserve' or 'react'.

In the current iteration of TypeScript 1.6 appears to have two options for --jsx, both preserve or react.

  • preserve will keep the jsx in the output. I presume this is so you can use tools like JSX to actually provide the translation.
  • react will remove the jsx syntax and turn it in to plain javascript so <div></div> in the TSX file would become React.createElement("div", null).

By passing the react option, here's where we end up:

> tsc --jsx react helloWorld.tsx
helloWorld.tsx(11,14): error TS2607: JSX element class does not support attributes because it does not have a 'props' property
helloWorld.tsx(11,44): error TS2304: Cannot find name 'mountNode'.

I'm going to tackle the last error next, as initially I didn't understand the JSX error above.

Fixing error TS2304: Cannot find name 'mountNode'.

This one I'll just make the compiler happy and presume we defined mountNode as an html element probably a <div id="mountNode"></div> somewhere in the global scope to keep this example short.

I place declare var mountNode: any; near the top of my helloWorld.tsx file and we're left with one last error:

> tsc --jsx react helloWorld.tsx
helloWorld.tsx(10,14): error TS2607: JSX element class does not support attributes because it does not have a 'props' property

Fixing error TS2607: JSX element class does not support attributes because it does not have a 'props' property

This last error is actually the one that had me stumped, and mostly why I'm writing this lengthy post so I hope you find it and can work through it a little quicker than it took me.

What's happening here is TypeScript is doing what it was intended to do. It's statically checking our JSX in this case.

If you look at our sample above where we call React.createClass(...) and compare that to the type definition we see: function createClass<P, S>(spec: ComponentSpec<P, S>): ClassicComponentClass<P>; you may notice P and S generic parameters to createClass<P, S> which I didn't supply earlier.

The naming here wasn't immediately obvious, but some snooping around in the type definitions and I eventually found out P is referring to the type we pass in defining the structure of the react props and S defines the state.

So in this Hello World example when we placed name="John" attribute inside the <HelloMessage name="John" /> element and since we didn't give a P or S to the React.createClass<P,S>(...), TypeScript was providing static type checking against an unknown type for P & S. In this case saying that we can't apply the attributes to the element because we did not provide a generic type P to define what props are allowed to be included.

To fix this I create a type by using an interface like below:

interface HelloWorldProps {
  name: string;
}

When I call React.createClass I pass in the HelloWorldProps interface for the props (P) and an any for the state (S) like so: React.createClass<HelloWorldProps, any>(...)

YAY IT COMPILES!!!

Compiling the below by using tsc --jsx react helloWorld.tsx

/// <reference path="./typings/react/react.d.ts" />
import React = __React;
declare var mountNode: any;

interface HelloWorldProps {
  name: string;
}

var HelloMessage = React.createClass<HelloWorldProps, any>({
  render: function() {
    return <div>Hello {this.props.name}</div>;
  }
});

React.render(<HelloMessage name="John" />, mountNode);

We get the following output in helloWorld.js

/// <reference path="./typings/react/react.d.ts" />
var React = __React;
var HelloMessage = React.createClass({
    render: function () {
        return React.createElement("div", null, "Hello ", this.props.name, React.createElement("div", null));
    }
});
React.render(React.createElement(HelloMessage, {"name": "John"}), mountNode);

Let's improve it now...

Since we started with a plain JavaScript version of our sample and we're now using TypeScript we get to take advantage of some of what TypeScript brings to the table.

But before we do this, let's first break our code (from a compiler) perspective to see what the above gave us...

Let's break our example on purpose to see how TypeScript responds?

I changed one character in two places in the working helloWorld.tsx file and when I run the compiler I get the following two errors. Can you spot what changed?

/// <reference path="./typings/react/react.d.ts" />
import React = __React;
declare var mountNode: any;

interface HelloWorldProps {
  Name: string;
}

var HelloMessage = React.createClass<HelloWorldProps, any>({
  render: function() {
    return <div>Hello {this.props.mane} <div></div></div>;
  }
});

React.render(<HelloMessage name="John" />, mountNode);

Output:

> tsc --jsx react helloWorld.tsx
helloWorld.tsx(15,14): error TS2324: Property 'Name' is missing in type 'HelloWorldProps'.
helloWorld.tsx(15,28): error TS2339: Property 'name' does not exist on type 'HelloWorldProps'.

Did you spot the change made? If you did, amazing. If you didn't, don't feel bad - it's a very simple and easy error to make when writing plain javascript. One that can't be found without actually executing plain JS, debugging, running unit tests or other checkers before even finding the error.

If you didn't spot the change by looking directly at the code, can you spot the change by reading the compiler error output?

Ok, give up? - I changed the name to Name in the HelloWorldProps interface definition AND in the JSX this.props.mane I spelled the props name mane wrong (should be Name according to our interface definition). So then why did we only get the error on line 15 (that's the React.render(...) line).

TypeScript in this case is using the HelloWorldProps interface and it's definition to type-check the attributes used in the JSX <HelloMessage />.

This is great, the compiler found the error right in the JSX before we even tried to execute the code.

Why didn't it detect the mane mis-spelled variable?

I'm going to just take a guess on this one but I'm not a React guy yet, so it may have something to do with react internals (that I'm not feeling like digging into at the moment).

If you look at the react.d.ts you'll see that React.createClass<P,S>() returns a type of ClassicComponentClass<P>.

Thanks to a tip from Ryan (ya, the famous Ryan from the TypeScript team) has a great write up about TypeScript and JSX we should be avoiding all of the above use of React.createClass(...) and instead using the ES6 extends functionality which we can leverage in TypeScript.

Let's re-write...

Turning the HelloMessage variable into an ES6 class we now also get type checking inside the component on this.props options. YAY!!!:

/// <reference path="./typings/react/react.d.ts" />
import React = __React;
declare var mountNode: any;

interface HelloWorldProps extends React.Props<any> {
  Name: string;
}

class HelloMessage extends React.Component<HelloWorldProps, {}> {
  render() {
    return <div>Hello {this.props.mane}</div>;
  }
}

React.render(<HelloMessage name="John" />, mountNode);

The above gives us the following errors:

> tsc --jsx react helloWorld.tsx
helloWorld.tsx(11,35): error TS2339: Property 'mane' does not exist on type 'HelloWorldProps'.
helloWorld.tsx(15,14): error TS2324: Property 'Name' is missing in type 'HelloWorldProps'.
helloWorld.tsx(15,28): error TS2339: Property 'name' does not exist on type 'HelloWorldProps'.

Final Answer

So, a bit long winded, but below is the final sample HelloWorld React TypeScript JSX prototype.

/// <reference path="./typings/react/react.d.ts" />
import React = __React;
declare var mountNode: any;

interface HelloWorldProps extends React.Props<any> {
  name: string;
}

class HelloMessage extends React.Component<HelloWorldProps, {}> {
  render() {
    return <div>Hello {this.props.name}</div>;
  }
}

React.render(<HelloMessage name="John" />, mountNode);

Wrap-up

While it seemed a bit challenging getting started with TypeScript and JSX I could really see the benefit of the compiler helping out with React components going forward, and look forward to the future of this part of the project.

Thanks to the TypeScript team and community that helped bring all of this support together!

Happy TSXing!

Look at where DefinitelyTyped is Now

(Comments)

A Little Wondering

I installed Visual Studio 2015 the other night and was just poking around and stumbled upon the Cordova Project Template that is provided. This not necessarily that new, but is new to me as I spend most of my time these days working on a Mac doing iOS, Docker (for server-side stuff) and other projects. Though, I miss and still enjoy the benefits of Visual Studio for some projects.

While looking over the project template, I was surprised to see a familiar folder structure typings/cordova/* in the project.

Why is this folder structure familiar?

Because I created it almost 2 years ago when I threw together a PowerShell script that chucks DefinitelyTyped TypeScript definitions up onto NuGet.

Another example of the folder structure, while I can't say for certain, but I can only guess that when the winjs started converting over to TypeScript that they chose to put their type definitions into the same structure. I won't claim to have inspired it, but would be cool to say I did.

It was a bit surprising and humbling to discover Microsoft had in one of their default templates a reference to some TypeScript type definitions that (while I didn't have anything to do with the creation of the definitions, nor DefinitelyTyped) I did created the simple little utility that runs continuously up on TeamCity.CodeBetter.com and tirelessly updates NuGet packages as they change over on DefinitelyTyped.

This reminded me, that I hadn't checked up on the NuGet user account for DefinitelyTyped in quite a while so I decided to head over to the NuGet.org, take a glance at the account and check out how many downloads there had been?

DefinitelyTyped NuGet account

** HOLY Typings **, there've been over 2 million DT type definitions?

This little ditty was neat to find. I haven't done a ton with TypeScript in a while, but it's quite amazing to see the traction it is starting to gain.

This means that the TeamCity over at CodeBetter has pretty much been cranking out new and updated DefinitelyTyped TypeScript definitions for almost the last two years, but it's time to re-visit...

Thanks TeamCity @ CodeBetter

Before all of the new, cool, and hip online continuous integration systems came to be, one of the only free options for open source was an instance of JetBrains TeamCity over at CodeBetter and I'd like to throw out a big thank you for those at CodeBetter who put the time and effort into hosting this service for many of the .Net OSS projects out there.

Hello AppVeyor

I recently ported this job to an AppVeyor build for the DT project and after a quick round of using their wonderful support to work through a hurdle, I completed a new build at AppVeyor that replicated the one originally hosted on the TeamCity server.

So a HUGE thanks to AppVeyor for creating such great product, and offering free builds to the OSS community.

I'll keep an eye on the new build for a while (before I forget about it) and hopefully it'll run for another long time to come.

Happy TypeScripting!

How to compile plain *.js (JavaScript) files with the TypeScript Compiler

(Comments)

Challenge

Let's say you've been tasked with researching what porting your existing JavaScript solution to TypeScript may look like or maybe you're curious what this process would be like before you propose it to a lead or project manager.

The first hurdle you may stumble upon is - the tsc (TypeScript compiler) requires all of your files end with a .ts file extension. This can make quickly prototyping a port challenging. To get an idea of what a port to TypeScript will look like you don't want to deal with first renaming all of your files to .ts. Especially since there are probably files you want to not rename to TypeScript (like jQuery or AngularJS etc.

And since:

TypeScript is a typed superset of JavaScript that compiles to plain JavaScript.

it seems reasonable that you could acquire the benefits of the TypeScript compiler for even your existing JavaScript codebase.

Challenge Accepted.

The TypeScript compiler is open source, so let's take a dive into the compiler to see if this is something we can work around...

What are the steps we need to accomplish to make the compiler accept plain .js files?

  1. Get It
  2. Copy it
  3. Hack it
  4. Use it

Get it

Before we can get too far, let's first install the compiler onto our system.

I'll use npm to install typescript

npm install -g typescript

Note the -g here tells npm to install typescript globally. This adds the TypeScript compiler to your PATH so you can get right at tsc'ing your TypeScript code.

Copy it

We probably don't want to modify the globally installed version of tsc so we'll create a copy on our system to play around with. But, before we can do that we need to find where it is.

If on Windows or Mac

where tsc

NOTE: if you're using the PowerShell console on windows be sure to type out where.exe tsc because where is aliased to Where-Object in PowerShell which won't help us out in this case.

Once you've found the path to your version of tsc

Mine was in

  • C:\Users\jason\AppData\Roaming\npm\tsc
  • C:\Users\jason\AppData\Roaming\npm\tsc.cmd

Look at the contents of the tsc.cmd for Windows and tsc for non Windows machines. You'll notice that they are essentially executing node.exe passing in an argument to another tsc file in the node_modules path.

Take the two tsc and tsc.cmd files, copy them into a working folder MyJSCompiler and rename them. I named mine jsc and he shall be my squishy. Then take the contents of the node_modules/typescript/* folder (and path structure) and copy them to your working directory.

When you're done you should have a directory that looks something like this


--MyJSCompiler
  |   jsc                 <-- notice the re-named file from tsc -> jsc
  |   jsc.cmd             <-- notice the re-named file from tsc.cmd -> jsc.cmd
  |   
  ----node_modules
      ----.bin
      |       tsc
      |       tsc.cmd
      |       
      ----typescript
          |   .npmignore
          |   CopyrightNotice.txt
          |   LICENSE.txt
          |   package.json
          |   README.txt
          |   ThirdPartyNoticeText.txt
          |   
          ----bin
              |   lib.d.ts
              |   tsc
              |   tsc.js
              |   typescript.js
              |   
              ----resources
                  |   diagnosticMessages.generated.json
                  |   
                  ----(*.json files excluded for brevity)

Now you should be able to call your local version of jsc at the command line.

Hack it

Since we have a local version we can hack on now, let's find out what needs to change. Thanks to Ryan for already giving us a clue here.

Open up the node_modules/typescript/bin/tsc.js file (in our working directory from the previous step) and apply the below diff/changes.

     function isTSFile(fname) {
-        return isFileOfExtension(fname, ".ts");
+        return isFileOfExtension(fname, ".ts") || isFileOfExtension(fname, ".js");
     }

We've now implemented a small tweak to the TypeScript compiler that allows us to compile plain JavaScript files.

Use it

I won't be going into how to use the tsc compiler in this post, but you can now call the compiler passing along your *.js files and begin learning what changes you will need to make to your existing codebase to satisfy the compiler.

Be sure to use the --out FILE or --outDir DIRECTORY options because if you don't the compiler will take the input javascript file and overwrite it with it's compiled version.

!!WARNING!! I'll say that again, to use it, be sure to use the --out FILE or --outDir DIRECTORY options because if you don't the compiler will take the input javascript file and overwrite the original with it's compiled version.

With source control, this can potentially be a fun experiment to see what TypeScript's version looks compared to your own, but I'll leave that up to you to play with.

Best of luck on your port to TypeScript.

Introducing ToTypeScriptD - Automatic TypeScript Definition files for C++/CX or .Net assemblies

If you’re unfamiliar with TypeScript, I’d highly recommend checking it out. It’s my new preferred way to write JavaScript based applications.

If you write an application in TypeScript and need to interact with an external set of either JavaScript or C++/CX dependencies then you will quickly find that you are going to be mucking with some TypeScript Definition files and the more you take a dependency on the WinRT runtime the more type definitions you probably need to manually create.

It’s true that the TypeScript team has created a set of TypeScript Definition files for WinRT and WinJS, but I couldn’t get any comment as to how they were generated. And the more I looked into what was provided, the more I realized that there were inconsistencies and inaccuracies that were not good. Which led me to believe that they weren’t working on a super-secret tool to automatically generate these for us.

What is ToTypeScriptD ?

ToTypeScriptD is a quick little attempt at automatically taking the type system provided to us by a C++/CX winmd or .Net assembly file and projecting that into a set of TypeScript Definition files. This tool can basically take anything that is Ecma 355 Common Language Infrastructure (CLI) and spit out a .d.ts file for you automatically.

Why would I need this?

In the project readme I describe two use-cases that I know of (so far).

  1. If you build a 'Modern' (come on, we still call it Metro) Windows 8 app with WinJS and want to leverage TypeScript, wouldn't it be nice to get a set of TypeScript Definition files that reflect the native API's you're calling in the platform without manually creating the definition files?
  2. Say your building an MVC/WebAPI server application. It would be useful if your C.I./Build system could generate a set of TypeScript interfaces that were based on the server objects used to render your JSON API. This can be useful for client side JavaScript/TypeScript libraries that need to consume these objects and also provide a simple way to document the format of the input and output API response objects.

How do I get the project and start using it?

Before you install the tool and dive in head first, I would like warn you that I have only been working on this for about 2 weeks (in my off-hours) and there is going to be lots of room for improvement. But it’s currently generating a solid set of .d.ts files for .winmd files.

Head over to the project home page and check out the Readme for some quick-start information.

I’d would love to hear feedback. If anyone is interested in helping out, you can submit new issues or review the open issues list and start submitting pull requests.

Happy TypeScripting!

TypeScript Presentation at the Northern Nevada Software Developers Group (NNSDG)

This past week I gave a talk on TypeScript which was fun and well received.

If you have any interest in the slides they are hosted up on GitHub

Or if you just want to look at the talk in MarkDown

The markdown page is useful for a quick lookup on syntax if you forgot how something should look. I’d like to continue to evolve the presentation, so if you have any feedback or suggestions, please feel free to open a pull request and start a conversation.

Thanks to those who attended!