• Fable: React you can be proud of ! Part 2: Optimizing React

    To optimize React we’ll need to dig into how it works and I’ll show a series of specific optimizations to use.

    The post is part of a series on Fable / React optimizations split in 3 parts:

    1. React in Fable land
    2. Optimizing React (This one)
    3. Applying to Elmish

    How does React work

    The mechanism is described in a lot of details on the Reconciliation page of React documentation, I won’t repeat the details but essentially React Keep a representation of what it’s element tree is currently, and each time a change need to propagate it will evaluate what the element tree should now be and apply the diff.

    A few rules are important for performance:

    • Change of element type or component type makes React abandon any DOM diffing and the old tree is destroyed in favor of the new one.
      • This should be a rare occurrence, any big DOM change is pretty slow as it will involve a lot of destruction / creation by the browser and a lot of reflowing.
      • On the other hand we want this to happen if we know that the HTML elements under some Component will drastically change: no need to force React to diff 100s of elements if we know that it’s a very different page that is shown. The diff also has a price.
    • React mostly compares elements in order so adding an element at the start of a parent will change ALL children. The key attribute should be used to override this behaviour for anything that is more or less a list of elements.
      • Keys should also be stable, an array index for example is a pretty bad key as it’s nearly what React does when there are no keys.
    • The fastest way to render a component is to not render it at all, so shouldComponentUpdate (And PureComponent that is using it) are the best tools for us.
    • Functional components are always rendered, but they are cheaper than normal ones as all the lifetime methods are bypassed too.

    We can roughly consider for optimization purpose that each component in the tree is in one of these 4 states after each change (ordered from better performance-wise to worse) :

    1. Not considered (Because its parent wasn’t re-rendered) ❄️❄️
    2. Returned false to shouldComponentUpdate ❄️
    3. Render was called but returned the same tree as before 🔥
    4. Render was called but the tree is different and the document DOM need to be mutated 🔥🔥

    PureComponent

    The first optimization that is especially useful for us in F# is the PureComponent. It’s a component that only updates when one of the elements in its props or state changed and the comparison is done in a shallow way (by comparing references).

    It’s ideal when everything you manipulate is immutable, you know, like F# records 😉.

    Let’s take a small sample to see how it’s good for us. Test it as-is and with the Component replaced with a PureComponent :

    type Canary(initialProps) =
        inherit Component<obj, obj>(initialProps) // <-- Change to PureComponent here
        let mutable x = 0
        override this.render() =
            x <- x + 1
            div [] [ofInt x; str " = "; str (if x > 1 then "☠️" else "🐤️") ]
    
    let inline canary () = ofType<Canary,_,_> createEmpty []
    
    type [<Pojo>] CounterState = { counter: int }
    
    type Counter(initialProps) as this =
        inherit Component<obj, CounterState>(initialProps)
        do
            this.setInitState({ counter = 0})
    
        let add = this.Add
    
        member this.Add(_:MouseEvent) =
            this.setState({ counter = this.state.counter + 1 })
    
        override this.render() =
            div [] [
                canary ()
                div [] [ str "Counter = "; ofInt this.state.counter ]
                button [OnClick add] [str "👍"]
            ]
    
    let inline counter () = ofType<Counter,_,_> createEmpty []
    
    let init() =
        ReactDom.render(counter (), document.getElementById("root"))
    

    Before: Dead Canary

    After: Living Canary

    While our canary has no reason to update, each time the button is clicked it will actually re-render. But as soon as we convert it to a PureComponent it’s not updating anymore: none of its props or state change so react doesn’t even call render().

    Beware: Passing functions

    If you look in the previous samples, each time I pass a function it’s never a lambda declared directly in render() or even a member reference but it’s a field that points to a member.

    The reason for that is that for react to not apply changes for DOM elements or for PureComponent the references must be the same and lambdas are re-recreated each time so their reference would be different.

    But members? Members stay the same so we should be able to pass this.Add and have it work. But JavaScript is a weird language where passing this.Add would pass the method add without any this attached, so to keep the semantic of the F# language Fable helpfully do it for us and transpiles it to this.Add.bind(this) instead. But this also re-creates a reference each time so we must capture the bound version in a variable during the construction of the object.

    It’s hard to prove it with the button so let’s prove it by moving the button creation to our lovely 🐤:

    type [<Pojo>] CanaryProps = { add: MouseEvent -> unit }
    
    type Canary(initialProps) =
        inherit PureComponent<CanaryProps, obj>(initialProps)
        let mutable x = 0
        override this.render() =
            x <- x + 1
            div [] [
                button [OnClick this.props.add] [str "👍"]
                span [] [ofInt x; str " = "; str (if x > 1 then "☠️" else "️️️🐤️") ]
            ]
    
    let inline canary props = ofType<Canary,_,_> props []
    
    type [<Pojo>] CounterState = { counter: int }
    
    type Counter(initialProps) as this =
        inherit Component<obj, CounterState>(initialProps)
        do
            this.setInitState({ counter = 0})
    
        let add = this.Add
    
        member this.Add(_:MouseEvent) =
            this.setState({ counter = this.state.counter + 1 })
    
        override this.render() =
            div [] [
                canary { add = add }
                canary { add = this.Add }
                canary { add = (fun _ -> this.setState({ counter = this.state.counter + 1 })) }
                div [] [ str "Counter = "; ofInt this.state.counter ]
            ]
    
    let inline counter () = ofType<Counter,_,_> createEmpty []
    
    let init() =
        ReactDom.render(counter (), document.getElementById("root"))
    

    Only the first canary live

    Using toArray/toList and refs

    It’s tempting in F# to use list expressions to build React children even when we have lists as it allows for a very nice syntax, but it can be a performance problem and force useless renders. Let’s see a problematic sample:

    type [<Pojo>] CanaryProps = { name: string }
    
    type Canary(initialProps) =
        inherit PureComponent<CanaryProps, obj>(initialProps)
        let mutable x = 0
        override this.render() =
            x <- x + 1
            div [] [str (if x > 1 then "☠️" else "️️️🐤️"); str " "; str this.props.name ]
    
    let inline canary props = ofType<Canary,_,_> props []
    
    let goodNames = ["Chantilly"; "Pepe"; "Lester"; "Pete"; "Baby"; "Sunny"; "Bluebird"]
    
    type [<Pojo>] CanariesState = { i: int; names: string list }
    
    type Counter(initialProps) as this =
        inherit Component<obj, CanariesState>(initialProps)
        do
            this.setInitState({ i = 0; names = [] })
    
        let add = this.Add
    
        member this.Add(_:MouseEvent) =
            let name = goodNames.[this.state.i % goodNames.Length]
            let names = name :: this.state.names
            this.setState({ i = this.state.i + 1; names = names })
    
        override this.render() =
            div [] [
                yield button [OnClick add] [str "🥚"]
                yield! this.state.names |> List.map(fun n -> canary { name = n })
            ]
    
    let inline counter () = ofType<Counter,_,_> createEmpty []
    
    let init() =
        ReactDom.render(counter (), document.getElementById("root"))
    

    Only the last canary live

    It seem that Chantilly survives but in fact it’s an illusion, a new element is always created at the end with his name, and all others are mutated.

    So let’s fix it by exposing an array via toList and assigning an unique key to all our canaries:

    type [<Pojo>] CanaryProps = { key: string; name: string }
    
    type Canary(initialProps) =
        inherit PureComponent<CanaryProps, obj>(initialProps)
        let mutable x = 0
        override this.render() =
            x <- x + 1
            div [] [str (if x > 1 then "☠️" else "️️️🐤️"); str " "; str this.props.name ]
    
    let inline canary props = ofType<Canary,_,_> props []
    
    let goodNames = ["Chantilly"; "Pepe"; "Lester"; "Pete"; "Baby"; "Sunny"; "Bluebird"]
    
    type [<Pojo>] CanariesState = { i: int; canaries: (int*string) list }
    
    type Counter(initialProps) as this =
        inherit Component<obj, CanariesState>(initialProps)
        do
            this.setInitState({ i = 0; canaries = [] })
    
        let add = this.Add
    
        member this.Add(_:MouseEvent) =
            let name = goodNames.[this.state.i % goodNames.Length]
            let canaries = (this.state.i,name) :: this.state.canaries
            this.setState({ i = this.state.i + 1; canaries = canaries })
    
        override this.render() =
            div [] [
                button [OnClick add] [str "🥚"]
                this.state.canaries
                    |> List.map(fun (i,n) -> canary { key = i.ToString(); name = n })
                    |> ofList
            ]
    
    let inline counter () = ofType<Counter,_,_> createEmpty []
    
    let init() =
        ReactDom.render(counter (), document.getElementById("root"))
    

    Every canary live

    We could have kept using yield! instead of using ofList and it would have worked here with only the keys but it’s better to always use ofList.

    By using it an array is passed to React and it will warn us on the console if we forget to use key.

    It also creates a new scope, avoiding problems if we wanted to show another list in the same parent with keys in common (duplicate keys under a same parent aren’t supposed to happen).

    Functional components in other modules

    This is more of a current Fable issue/bug that might be solved at some point than something coming from React but it can wreck performance completely.

    The problem is that while directly referencing a function like List.map foo will generate something like listMap(foo). Doing the same with a function that is exposed by a module List.map m.foo will generate listMap(foo.m.bind(foo)). It’s necessary when the target function is a javascript one as some of them require it but is useless for F# functions. Fable isn’t currently smart enough to differentiate them.

    Fable was actually doing it for all function creating the problem for all Functional components but I fixed it in a PR released as part of Fable 1.3.8

    This problem has exactly the same cause as the one evoked in Beware: Passing functions but it has a simple workaround: use the function via ofFunction in the same module in a non-inline function and expose that instead of the Functional Component itself.

    // The variable need to be extracted because what happens is worse
    // than just render() being called multiple time: The whole component
    // is re-created on each change !
    let mutable x = 0
    
    type Canary(initialProps) =
        inherit PureComponent<obj, obj>(initialProps)
        override this.render() =
            x <- x + 1
            div [] [ofInt x; str " = "; str (if x > 1 then "☠️" else "🐤️") ]
    
    let inline canary () = ofType<Canary,_,_> createEmpty []
    
    module WrapperModule =
        let Wrapper(props: obj) =
            div [] [
                h3 [] [str "In module"]
                canary ()
            ]
    
        // XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        // Remove 'inline' here and the problem is solved, MAGIC !
        // XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        let inline wrapper () = ofFunction Wrapper createEmpty []
    
    type [<Pojo>] CounterState = { counter: int }
    
    type Counter(initialProps) as this =
        inherit Component<obj, CounterState>(initialProps)
        do
            this.setInitState({ counter = 0})
    
        let add = this.Add
    
        member this.Add(_:MouseEvent) =
            this.setState({ counter = this.state.counter + 1 })
    
        override this.render() =
            div [] [
                WrapperModule.wrapper()
                div [] [ str "Counter = "; ofInt this.state.counter ]
                button [OnClick add] [str "👍"]
            ]
    
    let inline counter () = ofType<Counter,_,_> createEmpty []
    
    let init() =
        ReactDom.render(counter (), document.getElementById("root"))
    

    Before: Dead canary

    After: Living canary

    Letting React concatenate

    While it’s only important with very frequent updates a little detail that can be interesting to look at is how strings are concatenated. The 3 choices are (from better to worse performance):

    override this.render() =
        ul [] [
            // React will use it's DOM diffing and provide very fast update
            li [] [ str "Hello, "; str this.props.name]
    
            // Javascript concatenation is a also very fast
            li [] [ str ("Hello, " + this.props.name)]
    
            // sprintf is more complex and slower, it's perfectly fine for
            // elements that don't render very often but it's not free
            li [] [ str (sprintf "Hello, %s" this.props.name)]
        ]
    

    That’s all folks

    Not a lot more to say, avoid render(), don’t kill the 🐤️🐤️🐤️ in the Fable mine and everything will be fine !

    Next article will be on Elmish and how it ties with all of that.

  • Fable: React you can be proud of ! Part 1: React in Fable land

    Fable coupled with Fable.React and Fable.Elmish.React are powerful tools to generate javascript applications. But generating good and fast React code is an already complex task that isn’t made simpler by using a different language and a transpiler.

    In this series of posts I plan to show how some common React constructs are done in F# and what to look for when optimizing them.

    The posts will be:

    1. React in Fable land (This one)
    2. Optimizing React
    3. Applying to Elmish

    Starting a sample Fable React project

    If you want to try the code for yourself you’ll need a sample

    • Start with the fable template as in the Getting started guide
    • Replace the h1 and canvas tags in public/index.html with <div id="root"></div>
    • Ensure that we have the latest stable version of everything: .paket\paket.exe update
    • Add Fable.React using paket: .paket/paket.exe add Fable.React -p src/FableApp.fsproj
    • Add the necessary JS libs with yarn add react react-dom
    • Change the App.fs file to look like that:
    module FableApp
    
    open Fable.Core
    open Fable.Core.JsInterop
    open Fable.Import
    open Fable.Import.Browser
    open Fable.Import.React
    open Fable.Helpers.React
    open Fable.Helpers.React.Props
    
    let init() =
        let element = str "Hello 🌍"
        ReactDom.render(element, document.getElementById("root"))
    
    init()
    

    Hello 🌍

    Creating HTML elements

    As F# doesn’t have any JSX-like transform creating React elements is done as explained in the React Without JSX article, except that instead of directly using createElement a bunch of helpers are available in the Fable.Helpers.React module.

    For HTML elements the resulting syntax is strongly inspired by the Elm one.

    Here is a small sample of the more common ones :

    let element =
        // Each HTML element has an helper with the same name
        ul
            // The first parameter is the properties of the elements.
            // For html elements they are specified as a list and for custom
            // elements it's more typical to find a record creation
            [ClassName "my-ul"; Id "unique-ul"]
    
            // The second parameter is the list of children
            [
                // str is the helper for exposing a string to React as an element
                li [] [ str "Hello 🌍" ]
    
                // Helpers exists also for other primitive types
                li [] [ str "The answer is: "; ofInt 42 ]
                li [] [ str "π="; ofFloat 3.14 ]
    
                // ofOption can be used to return either null or something
                li [] [ str "🤐"; ofOption (Some (str "🔫")) ]
                // And it can also be used to unconditionally return null, rendering nothing
                li [] [ str "😃"; ofOption None ]
    
                // ofList allows to expose a list to react, as with any list of elements
                // in React each need an unique and stable key
                [1;2;3]
                    |> List.map(fun i ->
                        let si = i.ToString()
                        li [Key si] [str "🎯 "; str si])
                    |> ofList
    
                // fragment is the <Fragment/> element introduced in React 16 to return
                // multiple elements
                [1;2;3]
                    |> List.map(fun i ->
                        let si = i.ToString()
                        li [] [str "🎲 "; str si])
                    |> fragment []
            ]
    

    Output of helpers demonstration

    React components

    While it is possible to use React as a templating engine for HTML by using only built-in components, what really unlocks the power of React and where lies the biggest potential for optimization is in its user-defined components.

    Creating Components in F# is really similar to how they are created in modern JavaScript. The main difference comes when consuming them as we’ll use the ofType and ofFunction helpers (Instead of using JSX or React.createElement).

    Functional Components

    The easiest to use F# components are Functional ones, they don’t need a class, a simple function taking props and returning a ReactElement will do. They can then be created using the ofType helper.

    Let’s see how they are created in JavaScript:

    function Welcome(props) {
      return <h1>Hello, {props.name}</h1>;
    }
    
    function init() {
        const element = <Welcome name="🌍" />;
        ReactDOM.render(element, document.getElementById("root"));
    }
    

    And the equivalent in F#:

    type [<Pojo>] WelcomeProps = { name: string }
    
    let Welcome { name = name } =
        h1 [] [ str "Hello, "; str name ]
    
    let inline welcome name = ofFunction Welcome { name = name } []
    
    let init() =
        let element = welcome "🌍"
        ReactDom.render(element, document.getElementById("root"))
    

    Hello, 🌍

    Some notes:

    • We had to declare WelcomeProps while JavaScript could do without, and in addition we had to declare it as [<Pojo>] to ensure that Fable generates an anonymous JS object instead of creating a class (React rejects props passed as class instances)
    • Using sprintf in the F# sample could have seemed natural but using React for it is a lot better on a performance standpoint as we’ll see later.

    Note: Due to some peculiarities of the Fable transform there can be negative performance impact of using them but they are avoidable if you know what to look for. I’ll detail this some more in the second post

    Class Components

    To create an user-defined component in F# a class must be created that inherit from Fable.React.Component<'props,'state> and implement at least the mandatory render() method that returns a ReactElement.

    Let’s port our “Hello World” Component:

    type [<Pojo>] WelcomeProps = { name: string }
    
    type Welcome(initialProps) =
        inherit Component<WelcomeProps, obj>(initialProps)
        override this.render() =
            h1 [] [ str "Hello "; str this.props.name ]
    
    let inline welcome name = ofType<Welcome,_,_> { name = name } []
    
    let init() =
        let element = welcome "🌍"
        ReactDom.render(element, document.getElementById("root"))
    

    Hello, 🌍

    Nothing special here, the only gotcha is that the props passed in the primary constructor even though they are in scope in the render() method should not be used. It can be avoided at the price of a little more complex syntax:

    type Welcome =
        inherit Component<WelcomeProps, obj>
        new(props) = { inherit Component<_, _>(props) }
        override this.render() =
            h1 [] [ str "Hello "; str this.props.name ]
    

    Class Component with state

    All features of React are available in Fable and while the more “Functional” approach of re-rendering with new props is more natural using mutable state is totally possible :

    // A pure, stateless component that will simply display the counter
    type [<Pojo>] CounterDisplayProps = { counter: int }
    
    type CounterDisplay(initialProps) =
        inherit PureStatelessComponent<CounterDisplayProps>(initialProps)
        override this.render() =
            div [] [ str "Counter = "; ofInt this.props.counter ]
    
    let inline counterDisplay p = ofType<CounterDisplay,_,_> p []
    
    // Another pure component displaying the buttons
    type [<Pojo>] AddRemoveProps = { add: MouseEvent -> unit; remove: MouseEvent -> unit }
    
    type AddRemove(initialProps) =
        inherit PureStatelessComponent<AddRemoveProps>(initialProps)
        override this.render() =
            div [] [
                button [OnClick this.props.add] [str "👍"]
                button [OnClick this.props.remove] [str "👎"]
            ]
    
    let inline addRemove props = ofType<AddRemove,_,_> props []
    
    // The counter itself using state to keep the count
    type [<Pojo>] CounterState = { counter: int }
    
    type Counter(initialProps) as this =
        inherit Component<obj, CounterState>(initialProps)
        do
            this.setInitState({ counter = 0})
    
        // This is the equivalent of doing `this.add = this.add.bind(this)`
        // in javascript (Except for the fact that we can't reuse the name)
        let add = this.Add
        let remove = this.Remove
    
        member this.Add(_:MouseEvent) =
            this.setState({ counter = this.state.counter + 1 })
    
        member this.Remove(_:MouseEvent) =
            this.setState({ counter = this.state.counter - 1 })
    
        override this.render() =
            div [] [
                counterDisplay { CounterDisplayProps.counter = this.state.counter }
                addRemove { add = add; remove = remove }
            ]
    
    let inline counter props = ofType<Counter,_,_> props []
    
    // createEmpty is used to emit '{}' in javascript, an empty object
    let init() =
        let element = counter createEmpty
        ReactDom.render(element, document.getElementById("root"))
    

    Counter = 42

    Note: This sample uses a few react-friendly optimizations that will be the subject of the second post.

    That’s all folks

    Nothing special this time and for anyone that know both React and Fable there was not a lot of new information but we’ll expand it next time !

  • Adding our first feature to Ionide

    This post is part of the F# Advent Calendar in English 2017 organized by Sergey Tihon

    Ionide is now one of the most used F# editing experience but it’s pretty unique as it’s based on an easily extensible editors: Visual Studio Code.

    In the world of VSCode extensibility no arcane knowledge is required, the editor is open source with a well written code base (in TypeScript), the plugin API is easy to get into and most plugins are open source too. While the Javascript (Or TypeScript) language might not be to the taste of everyone, we have Fable allowing us to continue to write F# and run in in the Javascript VM.

    And Ionide is written in Fable, so we’ll use Ionide and VSCode to edit Ionide itself.

    Let’s implement a very simple feature and then move to a bigger one! The simple feature will be to add a button in the editor when .fsx files are edited to call the FSI: Send File command, sending the full file to F# interactive. The more complex one will be to run the file in the terminal.

    Getting started

    First you’ll need the prerequisites (VScode, F#, dotnet, yarn, node, …) then let’s checkout a commit without the feature, run a first full build and start code :

    git clone git@github.com:ionide/ionide-vscode-fsharp.git
    cd ionide-vscode-fsharp
    git checkout -b having_fun c0922fc
    build build
    code .
    

    Once this first full build is setup well start our developer loop in VS Code:

    • In the Integrated terminal (View > Integrated Terminal menu) we run build watch :
      Build watch running
    • In the debug view we select the Launch Only configuration:
      Launch only being selected
    • Press F5 to start a debug instance of VS Code.

    Once started our loop is simple :

    • Type new buggy code.
    • Wait for the terminal to show that it rebuilt it successfully
    • Move to the debug instance and press F1 to run Reload Window
    • Test
    • Repeat

    The loop is fast and while not as good as a browser with auto-reloading, pretty nice to use.

    A JSON-only feature

    Our first target is to get something like that:

    Final result

    And we actually don’t need any code to do it, as the command already exists the only thing we need to do is to to change the release/package.json file. And as added bonus it’s not something that is build but instead used as-is by Code so we don’t even need to wait for the watch build to test it, simply reloading the test instance window will allow us to see our changes.

    While the file contains the same field as any package.json file, it’s a section specific to Code that interest us : contributes. It contains all of the different things that the Ionide extensions declare that it’s able to provide and it’s well documented on VS Code website.

    The command we want to run is shown as FSI: Send File in the command palette but to run it we’ll need it’s ID and that can be found in the commands section.

    {
      "contributes": {
        "commands": [
          {
            "command": "fsi.SendFile",
            "title": "FSI: Send File"
          }
        ]
      }
    }
    

    We currently want to show it in the editor title menu bar that is documented as being the editor/title section inside menu, so let’s add it:

    {
      "contributes": {
        "menus": {
            "editor/title": [
              {
                "command": "fsi.SendFile",
                "when": "editorLangId == 'fsharp' && resourceExtname == '.fsx'",
                "group": "navigation"
              }
            ],
        }
      }
    }
    

    A few notes here:

    • command is the command we wants to run, that we found just before
    • when is the condition for it to appear, we want to be sure that the file is marked as F# (The language ID is auto detected but the user can also change it manually via the selector on the bottom right) and that it’s a script (Sending .fs files is possible but we don’t want to suggest it so prominently in the UI)
    • group is used to group related commands but navigation is special as it tell code to make the item visible directly as an icon instead of being a line in the ... menu (the default)

    Note: resourceExtname is a feature that was added to VS Code by Krzysztof Cieślak, the author of Ionide specifically for this change!

    We reload the window and…

    No icon

    We forgot the icon 😢

    Turns out that the icon is declared along with the command (And show everywhere the command is shown) instead of being part of the menu, so we’ll get back to the command definition and add the icon

    {
      "contributes": {
        "commands": [
          {
            "command": "fsi.SendFile",
            "title": "FSI: Send File",
            "icon":
            {
              "light": "./images/send-light.svg",
              "dark": "./images/send-dark.svg"
            }
          }
        ]
      }
    }
    

    The images need to be placed in the release/image folder and to start copying existing ones is good enough. There are 2 images to allow for small variations between light and dark themes, but icon can also be the path to a single image if the distinction isn’t needed.

    And finally our feature is here, we can test it and verify that it works as expected, job done 🙌.

    Final result

    Adding our own command

    Now let’s add another small feature: Running FSX scripts in the terminal

    Menu with run and send

    To do that well go back to package.json and add our new command:

    {
      "contributes": {
        "commands": [
          {
            "command": "fsharp.scriptrunner.run",
            "title": "F#: Run script",
            "icon":
            {
              "light": "./images/run-light.svg",
              "dark": "./images/run-dark.svg"
            }
          }
        ],
        "menus": {
          "commandPalette": [
            {
              "command": "fsharp.scriptrunner.run",
              "when": "editorLangId == 'fsharp' && resourceExtname == '.fsx'"
            }
          ],
           "editor/title": [
            {
              "command": "fsharp.scriptrunner.run",
              "when": "editorLangId == 'fsharp' && resourceExtname == '.fsx'",
              "group": "navigation"
              }
            ]
          ]
        }
      }
    }
    

    We’ll also need to add a new file to host our code in src/Components/ScriptRunner.fs and add it to the .fsproj file. (After this step the watch script might need to be restarted manually to pickup the new file)

    namespace Ionide.VSCode.FSharp
    
    open System
    open Fable.Core
    open Fable.Import.vscode
    open Fable.Import.Node
    
    module ScriptRunner =
        let private runFile () =
            printfn "Hello world"
    
        let activate (context: ExtensionContext) =
            commands.registerCommand(
                "fsharp.scriptrunner.run",
                runFile |> unbox<Func<obj,obj>>) |> context.subscriptions.Add
    

    Adding the call to activate in fsharp.fsx is the final touch and our new button will finally do something (Even if it’s only writing “Hello world” to the Debug Console)

    // ...
    Forge.activate context
    Fsi.activate context // <--- Line added here
    ScriptRunner.activate context
    
    { ProjectLoadedEvent = Project.projectLoaded.event
      BuildProject = MSBuild.buildProjectPath "Build"
      GetProjectLauncher = Project.getLauncher
      DebugProject = debugProject }
    

    Finally, we write some code

    Let’s start with the most simple addition, creating a terminal with a fixed name that run fsi.exe with the current file as single parameter

    let private runFile () =
        let scriptFile = window.activeTextEditor.document.fileName
        let terminal = window.createTerminal("script", Environment.fsi, [| scriptFile|])
        terminal.show ()
    

    It works but show a big problem with our approach as it closes immediately the terminal, before we can even read the results of executing our script.

    A solution to this problem is to run a shell (cmd.exe for windows) and use it’s capabilities instead of directly starting F# Interactive.

    Ideally we would use it’s arguments that allow to run commands but windows program arguments function differently than unix platforms: They are a single string and programs are free to parse them as they want. As NodeJS started on unix it’s API wants an array of parameters that are then encoded using the MSVCRT rules but cmd.exe parameter /C has a very special handling that doesn’t follow theses conventions and we can’t use it.

    But VSCode allow us to send simulated keystrokes to the terminal so we’ll use this

    let private runFile () =
        let scriptFile = window.activeTextEditor.document.fileName
        let terminal = window.createTerminal("script", "cmd.exe", [||])
        terminal.sendText(sprintf "\"%s\" \"%s\" && pause && exit" Environment.fsi scriptFile)
        terminal.show ()
    

    First try, working

    We now have a working minimal sample and can start to build from there:

    • We need to change our directory to the one of the script (the default is the root of the workspace) before running it
    • The fixed title become confusing pretty fast if we run multiple different scripts

    For both we might be tempted to use System.IO, but that’s not something that’s currently translated:

    ERROR in ./src/Components/ScriptRunner.fs
    G:/Code/_ext/ionide-fs/src/Components/ScriptRunner.fs(24,24): (24,67) error FABLE:
        Cannot find replacement for System.IO.Path::GetDirectoryName
     @ ./src/fsharp.fs 45:0-71
     @ ./src/Ionide.FSharp.fsproj
    

    The solution is to use the NodeJS Path API directly, the whole NodeJS API is provided by Fable in the Fable.Import.Node Nuget package that Ionide already import

    open Fable.Import.Node
    
    let private runFile () =
        let scriptFile = window.activeTextEditor.document.fileName
        let scriptDir = Path.dirname(scriptFile)
    
        let title = Path.basename scriptFile
        let terminal = window.createTerminal(title, "cmd.exe", [| "/K" |])
        terminal.sendText(sprintf "cd \"%s\" && \"%s\" \"%s\" && pause && exit" scriptDir Environment.fsi scriptFile)
        terminal.show ()
    

    Now let’s cleanup, add unix support and we’re ready to send a PR:

    namespace Ionide.VSCode.FSharp
    
    open System
    open Fable.Import.vscode
    open Fable.Import.Node
    
    module ScriptRunner =
        let private runFile (context: ExtensionContext) () =
            let scriptFile = window.activeTextEditor.document.fileName
            let scriptDir = Path.dirname(scriptFile)
    
            let (shellCmd, shellArgs, textToSend) =
                match Os.``type``() with
                | "Windows_NT" ->
                    ("cmd.exe",
                     [| "/Q"; "/K" |],
                     sprintf "cd \"%s\" && cls && \"%s\" \"%s\" && pause && exit" scriptDir Environment.fsi scriptFile)
                | _ ->
                    ("sh",
                     [||],
                     sprintf "cd \"%s\" && clear && \"%s\" \"%s\" && echo \"Press enter to close script...\" && read && exit" scriptDir Environment.fsi scriptFile)
    
            let title = Path.basename scriptFile
            let terminal = window.createTerminal(title, shellCmd, shellArgs)
            terminal.sendText(textToSend)
            terminal.show ()
    
        let activate (context: ExtensionContext) =
            commands.registerCommand(
                "fsharp.scriptrunner.run",
                runFile |> unbox<Func<obj,obj>>) |> context.subscriptions.Add
    

    Conclusion

    VS Code an Ionide are the perfect introduction for any programmer that wants to start customizing his tools so don’t hesitate.

    You’re missing an icon ? Some command would make your life easier ? Fork, Build, Contribute !

  • More F# colors in the terminal

    After releasing ColoredPrintf I started investigating color terminals, especially how some terminals can display more than the traditional 16 colors.

    While on windows console colors are changed using a call to the SetConsoleTextAttribute API, on unix XTerm and other terminals follow the way the VT100 physical terminal did it with special escape sequences that are sent to the terminal but change it’s behavior instead of being displayed.

    It is especially interesting as such terminals are becomming more present, even in the windows world. ConEmu support a lot even if it’s buggy, the venerable windows terminal can be told to interpret them and it’ll even soon gain True Color support too.

    Compatibility

    If you’re on windows, expect bugs everywhere but if you have a mac or a linux OS you’re good. In more details :

    • On windows ConEmu is still the best you can do, it’s buggy but usable. There is a little trick to enable more than 16 colors but it’s easy and need to be done only once per session, I’ll explain how to do it from code in Appendix A but you can simply display one of the ConEmu sample file with cmd /c type "%ConEmuBaseDir%\Addons\AnsiColors256.ans" and it’ll activate it.
    • On macOS (Tested on sierra) the integrated terminal can display 256 colors without problems but will translate any True Color command to 16 colors, giving a very ugly result. But iTerm2 is free works !
    • On linux the Gnome default terminal, XTerm and Konsole works flawlessly (XTerm adapt true color codes to 256 colors but it’s algorithm is pretty good).
    • Visual Studio Code integrated terminal can do most of them and as the REPL provided by the Ionide plugin uses it we can use the F# REPL directly there. But beware the windows version of the VSCode terminal is pretty buggy at the moment and often completely break when control codes are sent (Well, resizing it is enough to break it, you don’t need much 😁)

    Using F# to play with escape sequences

    Escape sequences are generated by printing the ESC (0x1B) character followed by at least another character.

    The most common ones start with the “CSI” sequence ESC[ followed by parameters and ending with a character designating the command. The full list is on Wikipedia or MSDN.

    Let’s try that in F# :

    let esc = string (char 0x1B)
    System.Console.WriteLine(esc + "[31;1m" + "Hello world" + esc + "[0m")
    

    First sample

    It displayed Hello World in bright red as m is the graphic command taking sub-commands separated by ;, 31 is the red color and 1 signal that we want bold/bright text. And at the end 0 is the reset sub-command, without it the terminal would stay red on all following lines.

    Using kprintf we can create a printf-like function and start implementing a few of the escape sequences :

    let csi = esc + "["
    let printsequencef f = Printf.kprintf (fun s -> System.Console.Write(esc + s)) f
    let printcsif f = Printf.kprintf (fun s -> System.Console.Write(csi + s)) f
    let selectGraphicRendition (gr: int list) =
        printcsif "%sm" (System.String.Join(";", gr |> Seq.map string))
    let resetColor() = selectGraphicRendition [0]
    let setForeground i = selectGraphicRendition([30 + i])
    let setBackground i = selectGraphicRendition([40 + i])
    

    256 colors is enough for everyone

    Until then we’ve done nothing very interesting, after all the .Net console already support changing the console color to bright red, nothing new under the sun. But what the .Net console or Windows native API doesn’t support is the 38 and 48 graphics sub-commands, extended foreground an background colors.

    let setExtendedForeground i = selectGraphicRendition [38; 5; i]
    let setExtendedBackground i = selectGraphicRendition [48; 5; i]
    let setForegroundRgb r g b = selectGraphicRendition [38; 2; r; g; b]
    let setBackgroundRgb r g b = selectGraphicRendition [48; 2; r; g; b]
    

    The first variant accept an index in a 256 colors table and the second specify the R, G and B components.

    While all the terminal emulators I tested support both of them the colors the display will vary, especially for the R;G;B True Color mode. Some like macOS default terminal are pretty weird in the sense that it support 256 color mode but interpolate any R;G;B code to 16 colors.

    Let’s see what the palete looks like :

    let extendedBlock i =
        for j in i..i+5 do
            setExtendedBackground j
            printf "  "
    
    for row in 0..5 do
        for b in 0..5 do
            extendedBlock (16 + 36*b + row*6)
            resetColor()
            printf " "
        printfn ""
    

    The result in VS Code (On macOS) :

    Palete

    We need to be true to our colors

    Now that we know how to draw with true colors we can draw pictures on the terminal :

    open System.Drawing
    
    let showBitmap path =
        let bitmap = Image.FromFile(path) :?> Bitmap
        for y in 0..bitmap.Size.Height-1 do
            for x in 0..bitmap.Size.Width-1 do
                let px = bitmap.GetPixel(x, y)
                setBackgroundRgb (int px.R) (int px.G) (int px.B)
                printf "  "
            resetColor()
            printfn ""
    

    And we can apply it to a 40x40px version of the FSharp logo :

    Logo

    The code for all of that can be found in a Gist.

    I didn’t test the windows insider release so can’t tell how well microsoft True Color support in the native console works but it’s the next thing I want to try.

    Appendix A: ConEmu color support

    ConEmu support more than the standard 16 colors but if you try it, it won’t work directly, it seem that maybe for compatibility reasons the support isn’t always on.

    Before

    To trigger it an application must first scroll the terminal far away at least once with the T command :

    let scrollUp (lines: int) = printcsif "%iS" lines
    let scrollDown (lines: int) = printcsif "%iT" lines
    scrollDown 9999
    setCursorPos 9999 1
    

    After that ConEnu will display what we expect :

    After

    Appendix B: Enabling escape codes in Windows default terminal

    While the windows terminal support escape codes, it doesn’t do it directly. They need to be enabled using SetConsoleMode API :

    #nowarn "9"
    module DllImports =
        open System.Runtime.InteropServices
        open Microsoft.FSharp.NativeInterop
    
        let INVALID_HANDLE_VALUE = nativeint -1
        let STD_OUTPUT_HANDLE = -11
        let ENABLE_VIRTUAL_TERMINAL_PROCESSING = 0x0004
    
        [<DllImport("Kernel32")>]
        extern void* GetStdHandle(int nStdHandle)
    
        [<DllImport("Kernel32")>]
        extern bool GetConsoleMode(void* hConsoleHandle, int* lpMode)
    
        [<DllImport("Kernel32")>]
        extern bool SetConsoleMode(void* hConsoleHandle, int lpMode)
    
        let enableVTMode() =
            let handle = GetStdHandle(STD_OUTPUT_HANDLE)
            if handle <> INVALID_HANDLE_VALUE then
                let mode = NativePtr.stackalloc<int> 1
                if GetConsoleMode(handle, mode) then
                    let value = NativePtr.read mode
                    let value = value ||| ENABLE_VIRTUAL_TERMINAL_PROCESSING
                    SetConsoleMode(handle, value)
                else
                    printfn "no get"
                    false
            else
                printfn "no handle"
                false
    
    DllImports.enableVTMode() |> ignore
    
  • ColoredPrintf released

    ColoredPrintf is a very simple library that provide a printf-like function with color support. A first version of the library is available on NuGet with it’s source code on my Github.

    The syntax to use for color is $foreground;background[text] with the color names being the same as in the ConsoleColor enumeration.

    The main entry points of the library are the colorprintf and colorprintfn functions that have the same signature as printf and printfn (It uses my MasterOfFoo library for that).

    Example

    colorprintfn "Hello $red[world]."
    colorprintfn "Hello $green[%s]." "user"
    colorprintfn "$white[Progress]: $yellow[%.2f%%] (Eta $yellow[%i] minutes)" 42.33 5
    colorprintfn "$white;blue[%s ]$black;white[%s ]$white;red[%s]" "La vie" "est" "belle"
    

    result