• More F# colors in the terminal

    After releasing ColoredPrintf I started investigating color terminals, especially how some terminals can display more than the traditional 16 colors.

    While on windows console colors are changed using a call to the SetConsoleTextAttribute API, on unix XTerm and other terminals follow the way the VT100 physical terminal did it with special escape sequences that are sent to the terminal but change it's behavior instead of being displayed.

    It is especially interesting as such terminals are becomming more present, even in the windows world. ConEmu support a lot even if it's buggy, the venerable windows terminal can be told to interpret them and it'll even soon gain True Color support too.


    If you're on windows, expect bugs everywhere but if you have a mac or a linux OS you're good. In more details :

    • On windows ConEmu is still the best you can do, it's buggy but usable. There is a little trick to enable more than 16 colors but it's easy and need to be done only once per session, I'll explain how to do it from code in Appendix A but you can simply display one of the ConEmu sample file with cmd /c type "%ConEmuBaseDir%\Addons\AnsiColors256.ans" and it'll activate it.
    • On macOS (Tested on sierra) the integrated terminal can display 256 colors without problems but will translate any True Color command to 16 colors, giving a very ugly result. But iTerm2 is free works !
    • On linux the Gnome default terminal, XTerm and Konsole works flawlessly (XTerm adapt true color codes to 256 colors but it's algorithm is pretty good).
    • Visual Studio Code integrated terminal can do most of them and as the REPL provided by the Ionide plugin uses it we can use the F# REPL directly there. But beware the windows version of the VSCode terminal is pretty buggy at the moment and often completely break when control codes are sent (Well, resizing it is enough to break it, you don't need much 😁)

    Using F# to play with escape sequences

    Escape sequences are generated by printing the ESC (0x1B) character followed by at least another character.

    The most common ones start with the "CSI" sequence ESC[ followed by parameters and ending with a character designating the command. The full list is on Wikipedia or MSDN.

    Let's try that in F# :

    let esc = string (char 0x1B)
    System.Console.WriteLine(esc + "[31;1m" + "Hello world" + esc + "[0m")

    First sample

    It displayed Hello World in bright red as m is the graphic command taking sub-commands separated by ;, 31 is the red color and 1 signal that we want bold/bright text. And at the end 0 is the reset sub-command, without it the terminal would stay red on all following lines.

    Using kprintf we can create a printf-like function and start implementing a few of the escape sequences :

    let csi = esc + "["
    let printsequencef f = Printf.kprintf (fun s -> System.Console.Write(esc + s)) f
    let printcsif f = Printf.kprintf (fun s -> System.Console.Write(csi + s)) f
    let selectGraphicRendition (gr: int list) =
        printcsif "%sm" (System.String.Join(";", gr |> Seq.map string))
    let resetColor() = selectGraphicRendition [0]
    let setForeground i = selectGraphicRendition([30 + i])
    let setBackground i = selectGraphicRendition([40 + i])

    256 colors is enough for everyone

    Until then we've done nothing very interesting, after all the .Net console already support changing the console color to bright red, nothing new under the sun. But what the .Net console or Windows native API doesn't support is the 38 and 48 graphics sub-commands, extended foreground an background colors.

    let setExtendedForeground i = selectGraphicRendition [38; 5; i]
    let setExtendedBackground i = selectGraphicRendition [48; 5; i]
    let setForegroundRgb r g b = selectGraphicRendition [38; 2; r; g; b]
    let setBackgroundRgb r g b = selectGraphicRendition [48; 2; r; g; b]

    The first variant accept an index in a 256 colors table and the second specify the R, G and B components.

    While all the terminal emulators I tested support both of them the colors the display will vary, especially for the R;G;B True Color mode. Some like macOS default terminal are pretty weird in the sense that it support 256 color mode but interpolate any R;G;B code to 16 colors.

    Let's see what the palete looks like :

    let extendedBlock i =
        for j in i..i+5 do
            setExtendedBackground j
            printf "  "
    for row in 0..5 do
        for b in 0..5 do
            extendedBlock (16 + 36*b + row*6)
            printf " "
        printfn ""

    The result in VS Code (On macOS) :


    We need to be true to our colors

    Now that we know how to draw with true colors we can draw pictures on the terminal :

    open System.Drawing
    let showBitmap path =
        let bitmap = Image.FromFile(path) :?> Bitmap
        for y in 0..bitmap.Size.Height-1 do
            for x in 0..bitmap.Size.Width-1 do
                let px = bitmap.GetPixel(x, y)
                setBackgroundRgb (int px.R) (int px.G) (int px.B)
                printf "  "
            printfn ""

    And we can apply it to a 40x40px version of the FSharp logo :


    The code for all of that can be found in a Gist.

    I didn't test the windows insider release so can't tell how well microsoft True Color support in the native console works but it's the next thing I want to try.

    Appendix A: ConEmu color support

    ConEmu support more than the standard 16 colors but if you try it, it won't work directly, it seem that maybe for compatibility reasons the support isn't always on.


    To trigger it an application must first scroll the terminal far away at least once with the T command :

    let scrollUp (lines: int) = printcsif "%iS" lines
    let scrollDown (lines: int) = printcsif "%iT" lines
    scrollDown 9999
    setCursorPos 9999 1

    After that ConEnu will display what we expect :


    Appendix B: Enabling escape codes in Windows default terminal

    While the windows terminal support escape codes, it doesn't do it directly. They need to be enabled using SetConsoleMode API :

    #nowarn "9"
    module DllImports =
        open System.Runtime.InteropServices
        open Microsoft.FSharp.NativeInterop
        let INVALID_HANDLE_VALUE = nativeint -1
        let STD_OUTPUT_HANDLE = -11
        extern void* GetStdHandle(int nStdHandle)
        extern bool GetConsoleMode(void* hConsoleHandle, int* lpMode)
        extern bool SetConsoleMode(void* hConsoleHandle, int lpMode)
        let enableVTMode() =
            let handle = GetStdHandle(STD_OUTPUT_HANDLE)
            if handle <> INVALID_HANDLE_VALUE then
                let mode = NativePtr.stackalloc<int> 1
                if GetConsoleMode(handle, mode) then
                    let value = NativePtr.read mode
                    let value = value ||| ENABLE_VIRTUAL_TERMINAL_PROCESSING
                    SetConsoleMode(handle, value)
                    printfn "no get"
                printfn "no handle"
    DllImports.enableVTMode() |> ignore
  • ColoredPrintf released

    ColoredPrintf is a very simple library that provide a printf-like function with color support. A first version of the library is available on NuGet with it's source code on my Github.

    The syntax to use for color is $foreground;background[text] with the color names being the same as in the ConsoleColor enumeration.

    The main entry points of the library are the colorprintf and colorprintfn functions that have the same signature as printf and printfn (It uses my MasterOfFoo library for that).


    colorprintfn "Hello $red[world]."
    colorprintfn "Hello $green[%s]." "user"
    colorprintfn "$white[Progress]: $yellow[%.2f%%] (Eta $yellow[%i] minutes)" 42.33 5
    colorprintfn "$white;blue[%s ]$black;white[%s ]$white;red[%s]" "La vie" "est" "belle"


  • First release for MasterOfFoo

    MasterOfFoo is an F# library that intend to facilitate writing functions that act like the native printf one does.

    I presented it as an experiment during september's Paris F# Meetup (See the summary in french) but I feel like it's now ready to be tested by more people.

    It is now available on nuget as a preview version of 0.1.2 and the source code is on GitHub.

    The API is described in more details in the readme but here is a small taste of the possibilities with a proof-of-concept sqlCommandf implementation :

    module SqlCommandBuilder =
        open System.Text
        open BlackFox.MasterOfFoo
        open System.Data.Common
        open System.Data.SqlClient
        type internal SqlEnv<'Result, 'cmd when 'cmd :> DbCommand>(command: 'cmd, k) =
            inherit PrintfEnv<unit, unit, 'Result>(())
            let queryString = StringBuilder()
            let mutable index = 0
            override __.Finalize(): 'Result =
                command.CommandText <- queryString.ToString()
            override __.Write(s : PrintableElement) =
                let asPrintf = s.FormatAsPrintF()
                match s.ElementType with
                | PrintableElementType.FromFormatSpecifier ->
                    let parameter =
                        if typeof<DbParameter>.IsAssignableFrom(s.ValueType) then
                            s.Value :?> DbParameter
                            let paramName = sprintf "@p%i" index
                            index <- index + 1
                            let parameter = command.CreateParameter()
                            parameter.ParameterName <- paramName
                            parameter.Value <- s.Value
                    ignore(queryString.Append parameter.ParameterName)
                    command.Parameters.Add parameter |> ignore
                | _ ->
                    ignore(queryString.Append asPrintf)
            override __.WriteT(()) = ()
        let sqlCommandf (format : Format<'T, unit, unit, SqlCommand>) =
            doPrintfFromEnv format (SqlEnv(new SqlCommand (), id))
        let sqlRunf (connection: SqlConnection) (format : Format<'T, unit, unit, unit>) =
            let execute (cmd: SqlCommand) = cmd.ExecuteNonQuery() |> ignore
            doPrintfFromEnv format (SqlEnv(connection.CreateCommand(), execute))
    // Create the parametrized query "SELECT * FROM users WHERE name=@p0"
    let cmd = sqlCommandf "SELECT * FROM users WHERE name=%s" "vbfox"
    sqlRunf connection "DELETE FROM users WHERE name=%s" "vbfox"
  • Suave in IIS - Hello world

    Suave is an open source F# library implementing a web server in a functional style. The source code is on GitHub and releases are available on NuGet like most .NET open source projects these days.

    You can find the full sample presented here as a Visual Studio 2015 project on my GitHub. I'm using Paket to get the Suave package but the standard NuGet UI would work as well (albeit slower).

    The minimal hello world is simply :

    open Suave
    let main argv =
        startWebServer defaultConfig (Successful.OK "Hello World!")

    startWebServer is a bloking function that take two parameters:

    • A SuaveConfig object defining the configuration, the default one opens a single non encrypted server on port 8083.
    • A WebPart representing everything that would be served. The type is actually an alias for HttpContext -> Async<HttpContext option> and suave provide a lot of filters & combinators to do standard things, like for example returning something different depending on the URL (Kinda expected for a web server).

    Maybe i'll blog more about the different combinators and how to combine them to reproduce a few common web server cases (Serving files, protecting access to some pages, ...) but as this post is about IIS integration i'll keep the hello world code and build from that.


    IIS is the Microsoft web server present by default on Windows Server & Desktops. If you don't already have it on your dev machine it's available from the Turn Windows features on or off in control panel. The entry is named Internet Information Services and you can either go with the defaults or get everything under there.

    Windows features

    Once installed, start IIS Manager and create a website listening on a random port on IP address with a Physical path corresponding to the bin/Debug directory of the hello world project.


    HttpPlatformHandler is an IIS addon that allow to use any http serving application where the executable is able to receive the port to listen to via an environment variable.

    To start the web site IIS run the executable with the HTTP_PLATFORM_PORT environment variable containing the port that it need to listen to and the module will then proxy requests to this port and sent back the response to the client. As the module insert itself inside the standard IIS processing it mean that things like logging, https certificates, respawning the process when an error happens or doing windows domain authentication can be handled by IIS without any change to the application code.

    To install it start the Web Platform Installer (Either from your start menu or from the right side actions in IIS Manager) and search for it.

    Web Platform Installer

    To be able to enable it per-site like we'll do in the next step, open IIS Manager and in the Feature View for the whole server, open Feature Delegation and enable Handlers Mappings as Read/Write.

    Using web.config

    IIS expect it's configuration in an XML file named web.config so you'll need to add an xml file named like that to your project with Copy to Output Directory configured to Copy if newer with the content :

    <?xml version="1.0" encoding="UTF-8"?>
                <remove name="httpplatformhandler" />
                <add name="httpplatformhandler"
                    resourceType="Unspecified" />
            <httpPlatform processPath="./SuaveIIS.exe"

    This assume that your exectutable is named SuaveIIS.exe so adapt accordingly.

    For this to work you'll also need to authorize IIS to read it, do to that open Explorer and change the Permissions on the bin folder to allow read & write access by the local IIS_IUSRS group.

    Getting the port to listen to from F# Code

    In the current state, HttpPlatformHandler will start your executable, kill it after 20s and restart it in a loop because it will never see it listening on it's specified port so we need to read the port from the environment variable :

    module IISHelpers =
        open System
        /// Port specified by IIS HttpPlatformHandler
        let httpPlatformPort =
            match Environment.GetEnvironmentVariable("HTTP_PLATFORM_PORT") with
            | null -> None
            | value ->
                match Int32.TryParse(value) with
                | true, value -> Some value
                | false, _ -> None

    And then when we detect that we're started from IIS (Because IISHelpers.httpPlatformPort is Some) we can configure Suave to listen locally on that port and nowhere else :

    open Suave
    let main argv =
        let config = defaultConfig
        let config =
            match IISHelpers.httpPlatformPort with
            | Some port ->
                { config with
                    bindings = [ HttpBinding.mkSimple HTTP "" port ] }
            | None -> config
        startWebServer config (Successful.OK "Hello World!")

    If you compile it and visit your IIS site in a browser you should see your Hello World being served by IIS.


    • If you can't compile because the executable file is already it mean that your executable is running in IIS. What you need to do is to remove the web.config file from the bin/Debug directory, IIS will take this as a hint that your configuration changed and that it need to stop using your binary.
  • Ode to LINQPad

    LINQPad is one of my favorite tools for C# development it allow for fast development of small bits of C# to prototype or answer StackOverflow questions.

    LINQPad UI

    The original feature was to allow easy scripting of databases by providing a LINQ to SQL view of all objects of a connection directly as members of the generated script class. But LINQPad is a lot more generic and can serve as a C# swiss army knife for a lot of utility scripts.

    Searching for NuGet packages

    The Free and Pro editions of LINQPad don't give you access to the ability to reference NuGet packages directly and the NuGet website doesn't provide any download link. We'll use LINQPad itself to be able to search and download any package by Id.

    The first thing we need to do is to get access to the list of packages. If possible not in the HTML tag-soup form...

    Turns out that NuGet API is partially OData-XML and partially some REST endpoint. As LINQPad natively support OData I'll use only that. To use it, add a new connection of type WCF Data Services pointing to https://www.nuget.org/api/v2/ and then create a C# Program query using this connection.

    To start exploring we'll select some data to look at what fields are available and how they look:

    void Main()

    Dump is a method that display an object in the output with a graphical representation. For tabular data (IQueryable, IEnumerable, ...) passing true as first argument will show it in a DataGrid.

    To search for a package we will want to filter by Id and get only the latest non-prerelease version so the fields we are interested in are IsAbsoluteLatestVersion and IsPrerelease in addition to Id :

    void Main()
        var search = Util.ReadLine("Id");
        var packages =
            from package in Packages
            where package.IsAbsoluteLatestVersion
                && !package.IsPrerelease
                && package.Id.StartsWith(search)
            select package;

    This version of the code also introduce Util.ReadLine a method that display an input line at the bottom of the LINQPad output.

    Adding download links

    Now that we have all info from our package we need the download URL, turns out that it isn't directly included in the data but we should be able to build it from the information we already have.

    As NuGet is open-source we can read the source code and find how package download urls are constructed :

    // Route to get packages
                       new { httpMethod = new HttpMethodConstraint("GET") },
                       context => CreatePackageService().DownloadPackage(context.HttpContext));

    We know version and packageId so we can now display the download URL.

    But LINQPad allow us to do a lot more: the Hyperlinq type is a magic one that when passed to Dump show a clickable link in the output window.

    The link can be constructed with an Action delegate that will be executed when clicked but for such a simple case we'll use the version taking directly an URL:

    foreach(var package in packages)
        var url = string.Format(@"https://www.nuget.org/api/v2/package/{0}/{1}",
            package.Id, package.Version);
        var text = string.Format(@"{0} (v{1})", package.Id, package.Version);
        new Hyperlinq(url, text).Dump();

    Which give us a nice clickable list of packages starting with a string directly in LINQPad output :

    Links in output


    Regarding the price for free you get the basic UI, the Pro versions is at 40$ (Autocompletion) and the Premium (Direct SQL table editing + NuGet) is at 75$.

    The price is small (and often discounted) but as Roslyn is Open Source and make creating IDE text editors with auto-completion a lot easier I expect movement in this space especially for the Pro version features.

  • Blog engine changed to Jekyll"

    This weekend I wanted to blog about libgit2sharp but I realized that my setup for blogging wasn't really practical. I used dotclear but by insisting on markdown and refusing the graphical editor it was only an hindrance for me.

    In response I investigated the domain of the simple blogging platforms running only locally and generating html files. The one I ended up using is a ruby tool named Jekyll; the one that is used by GitHub for their GitHub pages.

    What is nice is that using a web-hook on my blog repository the simple fact of pushing a new markdown file or modifying a sass file will re-generate all my blog with the changes.

  • Using CommandBindings in MVVM

    One sad thing in WPF currently is that the CommandBindings used to attach an action to a RoutedCommand don't support the MVVM pattern : Their properties can't be easily bound to the DataContext, and they don't support attaching a command represented by an ICommand.

    One of the solutions have always been to use the command sink that Josh Smith provided in it's CodePlex article Using RoutedCommands with a ViewModel in WPF. But it's syntax force multiple changes on the view and also force the model to implements the Execute and CanExecute independently out of an ICommand and don't begin to support the CanExecuteChanged event.

    My solution was to roll my own classes, that you could find in a Gist on GitHub to use it instead of a CommandBindings block in your UIElement you declare an attached property called Mvvm.CommandBindings and use it with nearly the same syntax :

        <Window x:Class="BlackFox.SampleWindow"
                         Target="{Binding MyCommandInViewModel}"
                         CanExecuteChangedSuggestRequery="True" />

    The only differences are that there is a Target (ICommand) property instead of the methods and that you could optionally force a CommandManager.InvalidateRequerySuggested(); when the command's CanExecuteChanged event is raised.

  • Using svndumpfilter to extract a folder in it's own repository

    What i'll explain here is how to use the svndumpfilter program to extract a folder of an SVN repository in it's own repository and make it disappear from the history of the original one.

    What you'll need

    You will need direct access to the server and to warn all your users that they'll need to checkout again everything you will move (if you don't renumber the revisions otherwise all users will need to checkout again any part of the repository they use)

    Knowing what to filter

    You will need to include all the folders you want along with all the folders where something moved, for example if you want /some/dir but it was at some point in the history named /somedir and renamed later you need to include both in the filter.

    The same apply if only some of the files moved from there in this case you need to include each file that moved.

    You'll end up with a list of files and directories representing all the history of the files that will be extracted.

    Dump the repository

    The first step is to dump the repository as svndumpfilter only act on dumps and not on directories themselves

    svnadmin dump repo > repo.dump

    Filter the history and load it in a new repository

    To create a dump that contains only the change in our chosen directories the command is :

    svndumpfilter include --drop-empty-revs --renumber-revs /some/dir /somedir < repo.dump > repo-only-some-dir.dump

    The two options --drop-empty-revs and --renumber-revs are what you'll need in 99% of the cases but anyway you'll either want both or none :

    • Both will give you a clean new repository with new revisions starting at zero and no empty revisions. It's clean but if you had anywhere else (in the source code comments, in your issue tracker or in some commit message) something referencing "r3628" it will now be incorrect.
    • None will give you a chance to have the same revision numbers (Only if you don't need to create new directories otherwise all revision numbers will move, see next paragraph)

    Then you'll need to create a new repository

    mkdir newrepo
    svnadmin create newrepo

    If some of the directories you included had a parent that wasn't himself included (like our /some/dir directory) you need to create them back

    svn checkout "file:////path/to/newrepo/" newrepocheckout
    mkdir newrepocheckout/some
    svn add newrepocheckout/some
    svn commit newrepocheckout -m "Prepare to load filtered history"
    rm -R newrepocheckout

    Then finally load your filtered history :

    svnadmin load newrepo < repo-only-some-dir.dump

    Remove the files from the originating repository history

    It's simplier as you normally won't have to create any directory, just load the dump in place of the old repo.

    The same choice is offered to you regarding removing the now-empty revs from the history and renumbering the other revisions but in this case it's more logical to keep the revisions as it will allow for all users that hadn't checked out the removed directories to continue to work with their checkouts. Otherwise they'll need a clean new checkout.

    rm -R repo
    svndumpfilter include /some/dir /somedir < repo.dump > repo-without-some-dir.dump
    mkdir repo
    svnadmin create repo
    svnadmin load repo < repo-without-some-dir.dump
  • SSL Observatory how to load the db

    The SSL Observatory is a great project from the EFF to gather all SSL certificates available on HTTPS websites and publish them for analysis. For more details see the SSL Observatory website.

    The main problem if you don't want to analyse it using their pre-configured Amazon EC2 configuration is that the compressed .sql file is around 3 Go compressed using LZMA and 3 To uncompressed so you can't even un-compress the file and need to pipe directly from 7-Zip to mysql.

    To do it either on windows or linux :

    7z e -so "observatory-dec-2010.sql.lzma" f | mysql -u root --password=toto42sh observatory2

    Note: The 7z tools is in the p7zip package on a debian system.

  • Things to do after formating a Sheevaplug

    Once a SheevaPlug is formated with sheevaplug-installer the minimum to do is :

    Set the MAC address

    Connect to the serial console, enter U-Boot and set the MAC address :

    setevn ethaddr 00:50:43:01:D2:94

    Set timezone and hostname

    Login and type :

    # Set the time-zone
    dpkg-reconfigure tzdata
    #Set the host name
    echo plugfox.vbfox.local > /etc/hostname
  • Gmail "mark as unread" now works correctly !

    As explained in the Gmail blog, the really annoying bug of the "mark as unread" button is now fixed.

    Before if you had a new post in a really old thread with dozens of mails and marked it unread, all the messages in the thread were marked as unread, expanding them all ! It was really annoying as for long threads it meant scrolling a lot to find the last mail really unread message.

    Good job, but now a "mark all messages after this one as unread" option on each message seem needed.