Irreal: Reproducible Research for Management

The utility of reproducible research and literate programming methods in engineering and science is clear and well known by now but we seldom consider how the techniques might be extended to other disciplines. If, however, you think about what you’re trying accomplish with those procedures—documentation of process, tracking of changes, the ability to reproduce your work product and conclusions, and a self-contained file containing all the relevant data and calculations—it’s clear that almost any endeavor could profit from applying these techniques.

Derek Feichtinger has an interesting post in which he describes the application of reproducible research and literate programming to management problems. As an example, he considers generating a budget for a pair of related projects. His workflow is to first generate an outline describing his goal and the information he has and to refine that with subheadings as more information becomes available. That provides a history of the project and automatically tracks changes.

Feichtinger has written some Elisp that extracts the relevant data from the outline and puts in table form. That table can be used for further calculations, graphs, and other output. The results can, of course, be exported in a variety of ways according to what is needed.

If you are in management or you have to produce reports for management, you should take a look at Feichtinger’s post. There are a lot of good ideas in it even if you don’t choose to follow his exact workflow. As Feichtinger says, using these techniques is far superior to the usual management practice of doing everything with Excel.

-1:-- Reproducible Research for Management (Post jcs)--L0--C0--May 23, 2018 03:33 PM

sachachua: 2018-05-21 Emacs news

Links from reddit.com/r/emacs, /r/orgmode, /r/spacemacs, Hacker News, planet.emacsen.org, YouTube, the changes to the Emacs NEWS file, and emacs-devel.

-1:-- 2018-05-21 Emacs news (Post Sacha Chua)--L0--C0--May 22, 2018 01:59 AM

scripter.co | Emacs: How do I write "Org mode"?

You write it just like that in the title — “Org” with capital “O”, and then lower-case “mode” separated by a space.

A super-quick intro to Org mode

Org mode is a fantastic major mode for Emacs, and people use it for all sorts of things like keeping notes, maintaining TODO lists, writing documentation, or even blogging—like I am doing here.

Inconsistency

Every now and then, I would see “Org mode” and related phrases written in the “wild” (like blogs, Reddit posts, tweets) as Org-mode, org-manual, org file, etc., with a mix-and-match of cases and hyphens.

So here is an attempt to familiarize more people with the documentation standard for referring to “Org” stuff. Below I am quoting the text from the official Org Documentation Standards:

  • Prefer “Org mode” to “Org-mode” or “org-mode”. This is simply because it reflects an existing convention in The Emacs Manual which consistently documents mode names in this form - “Text mode”, “Outline mode”, “Mail mode”, etc.
  • Likewise refer, if at all possible, to “Org file or “Org buffer” meaning with, great generality, any file or buffer which requires use of some part of Org to edit it properly.
  • Org uses “org-…” to ring fence a name space for itself in the Emacs code base. This is obviously retained in code snippets.

The right way

Org mode, Org manual, Org file

Only in the Elisp code, is it called org-mode, because it is a major mode, and has to be named so in code by convention, and for its separate org-* name space.

But for blogging, documentation, etc. –

    Org foo it is.

Reference

-1:-- How do I write "Org mode"? (Post Kaushal Modi)--L0--C0--May 21, 2018 08:35 PM

Irreal: Python Doctests, Literate Programming, and Org Mode

If you’re a Python programmer and like to have built-in tests for your functions, you probably know about doctest, a way of embedding one or more tests in the docstring of a Python function. Normally, they don’t do anything but if doctest is loaded as the main program or one of doctest’s methods are called, the tests are run and the results are printed to stdout.

John Kitchin likes the idea but found that the nit-picky formatting of the tests was annoying so he didn’t use them very often. Of course, being Kitchin he soon fixed that and in a clever way that leverages Org mode. Rather than fuss with the formatting of the tests, he just put a noweb marker where he wanted the test to be and then included the test as a normal Org mode code block later in the file. He added in a bit of Elisp to reformat the test code so that when the file is tangled it becomes a Python script that can be invoked in the normal way to run the tests.

Kitchin thinks of this as an example of literate programming, which it is, of course, but I prefer to think of it as another example of the power of embedding code in an Org file. The function tests can be run from the Org file to get the expected results for the tests, explanatory text can be added, and when you’re finished you just tangle it to an output file. That file can be invoked as a script to run the tests or imported into another Python file to access the functions in the normal way. Very impressive. Kitchin’s post includes a short video so you can see the system in action and get a better feel for how things work.

-1:-- Python Doctests, Literate Programming, and Org Mode (Post jcs)--L0--C0--May 21, 2018 03:40 PM

sachachua: 2018-05-14 Emacs news

Links from reddit.com/r/emacs, /r/orgmode, /r/spacemacs, Hacker News, planet.emacsen.org, YouTube, the changes to the Emacs NEWS file, and emacs-devel.

-1:-- 2018-05-14 Emacs news (Post Sacha Chua)--L0--C0--May 14, 2018 04:21 AM

Marcin Borkowski: Beeminder and Emacs

Today’s post is special – instead of writing it here, I’m a guest on the Beeminder blog. Head over there to read it! (And thanks to Beeminder folks for having me there!)
-1:-- Beeminder and Emacs (Post)--L0--C0--May 11, 2018 01:35 PM

scripter.co | Emacs: Accessing Devdocs from Emacs

Spoiled by being able to access in-built docs in Emacs at fingertips, here’s an attempt to kind-of do that for Nim documentation too, using devdocs.io.

Nim lang has good documentation for all its stdlib functions online. But the Emacs user in me does not like to switch back and forth between the Nim code in Emacs buffers and the docs outside in an external browser.

Well.. a solution to that, that this post is about, still needs one to look up the Nim docs in an external browser.. but the workflow is a bit better — You don’t need to manually launch the doc site, and you don’t need to then manually type in the search query.

If you want to skip the history and code analysis, you can directly jump to the Demo or the Final Code.

JSON Docs

I tried asking folks on r/nim if there was a good solution for in-editor Nim doc access. /u/PMunch from Reddit gave a wonderful solution—To generate JSON docs for the Nim stdlib, and then parse those to display the docs within Emacs.

    I would love that solution!

.. just that I don’t know how to get a nice single .json for the whole of Nim documentation.

If someone knows how to do that, please let me know.

Devdocs.io

So I continued my search online.. I was looking if someone had already implemented a way to access Nim docs from the command line, and that somehow led me to https://devdocs.io/nim/!

And after searching further for “devdocs Emacs”, I found these two Emacs packages:

Christopher Wellon’s devdocs-lookup

After reviewing the two packages, I decided to build my solution further upon the devdocs-lookup package1 by Christopher Wellons aka @skeeto from GitHub.

Here’s why —

  1. He first fetches the whole JSON search index from docs.devdocs.io for the picked “subject” (which would be “Nim” for this post). So the search index for Nim documentation would be at https://docs.devdocs.io/nim/index.json2.

    The retrieved JSON is then parsed using json-read.

        (defun devdocs-index (subject &optional callback)
          "Return the devdocs.io index for SUBJECT, optionally async via CALLBACK."
          (cl-declare (special url-http-end-of-headers))
          (let ((index (gethash subject devdocs-index))
                (url (format "%s/%s/index.json" devdocs-base-index-url subject)))
            (cond ((and index callback)
                   (funcall callback index))
                  ((and index (not callback))
                   index)
                  ((and (not index) (not callback))
                   (with-current-buffer (url-retrieve-synchronously url nil t)
                     (goto-char url-http-end-of-headers)
                     (setf (gethash subject devdocs-index) (json-read))))
                  ((and (not index) callback)
                   (url-retrieve
                    url
                    (lambda (_)
                      (goto-char url-http-end-of-headers)
                      (setf (gethash subject devdocs-index) (json-read))
                      (funcall callback (gethash subject devdocs-index))))))))
        

    Code Snippet 1: Function to fetch the search index from devdocs.io and parse the JSON
  2. The name and path properties from the parsed JSON are then stored in an association list here:

        (defun devdocs-entries (subject)
          "Return an association list of the entries in SUBJECT."
          (cl-loop for entry across (cdr (assoc 'entries (devdocs-index subject)))
                   collect (cons (cdr (assoc 'name entry))
                                 (cdr (assoc 'path entry)))))
        

    Code Snippet 2: Function to store the name and path properties to alists

    So a JSON entry like this:

    {
        "name": "os.walkDirRec",
        "path": "os#walkDirRec.i,string",
        "type": "os"
    }

    would translate to this Emacs-Lisp alist element:

    ("os.walkDirRec" . "os#walkDirRec.i,string")
  3. Then the list of all the car’s of such elements is used to create a collection of entries for completion:

        (defun devdocs-read-entry (subject)
          "Interactively ask the user for an entry in SUBJECT."
          (let ((names (mapcar #'car (devdocs-entries subject)))
                (hist (intern (format "devdocs--hist-%s" subject))))
            (unless (boundp hist)
              (set hist nil))
            (completing-read "Entry: " names nil :match nil hist)))
        

    Code Snippet 3: Function to show completion list based on "names" from the JSON-parsed database
  4. And finally, for the selected name, the associated path is retrieved from that alist, and we browse to that path using the Emacs browse-url function. User can of course configure the browser to be used when that function is called.

        (defun devdocs-lookup (subject entry)
          "Visit the documentation for ENTRY from SUBJECT in a browser."
          (interactive
           (let* ((subject (devdocs-read-subject))
                  (entry (devdocs-read-entry subject)))
             (list subject entry)))
          (let ((path (cdr (assoc entry (devdocs-entries subject)))))
            (when path
              (browse-url (format "%s/%s/%s" devdocs-base-url subject path))
              :found)))
        

    Code Snippet 4: Function to browse the doc page associated with the user-selected "name"

All of that worked beautifully. As I used it a few times though, I felt a need to add a touch of DWIM to that.

Making devdocs-lookup DWIM

Here are the 2 things that I wanted to happen automatically:

Auto-select subject based on major-mode if possible

In the original code, if I used devdocs-lookup function, I needed to manually select the “Nim” subject even when I called that function from a nim-mode buffer. At least for my use cases, I would want to access only Nim docs if I am looking up devdocs while in a Nim code buffer.

The package has an interesting function called devdocs-setup which would generate a function specific to each subject.. so for “Nim” subject, it would generate a devdocs-lookup-nim function.

But I wanted to avoid calling devdocs-setup too.

So below is what I did:

(defun devdocs-lookup (subject entry)
  "Visit the documentation for ENTRY from SUBJECT in a browser."
  (interactive
   (cl-letf (((symbol-function 'string-match-case-insensitive)
              (lambda (str1 str2)
                (string= (downcase str1) (downcase str2)))))
     (let* ((major-mode-str (replace-regexp-in-string "-mode" "" (symbol-name major-mode)))
            ;; If major mode is `nim-mode', the ("Nim" "nim") element
            ;; will be auto-picked from `devdocs-subjects'.
            (subject-dwim (cadr (assoc major-mode-str devdocs-subjects
                                       #'string-match-case-insensitive)))
            (subject (or subject-dwim (devdocs-read-subject)))
            (entry (devdocs-read-entry subject)))
       (list subject entry))))
  (let ((path (cdr (assoc entry (devdocs-entries subject)))))
    (when path
      (browse-url (format "%s/%s/%s" devdocs-base-url subject path))
      :found)))

Code Snippet 5: Modification of the original devdocs-lookup – Now auto-selects the subject if the major-mode matches.

Auto-filter using the symbol at point

The second thing that I wanted to work upon was to make Emacs kind of “know” what I was trying to search.

Originally, devdocs-read-entry would always show the completion-list pointing at the first entry. I wanted to make that a bit more intelligent.. If my point were on the walkDirRec proc identifier on a line like below,

for filepath in walkDirRec(appPath, yieldFilter={pcFile}):

I wanted the collection to narrow down to only the entries that matched “walkDirRec”.

Below is my modification to do that.

(defun devdocs-read-entry (subject)
  "Interactively ask the user for an entry in SUBJECT."
  (let ((names (mapcar #'car (devdocs-entries subject)))
        (hist (intern (format "devdocs--hist-%s" subject)))
        (init (symbol-name (symbol-at-point))))
    (unless (boundp hist)
      (set hist nil))
    (completing-read (format "Entry (%s): " subject) names nil :require-match init hist)))

Code Snippet 6: Modification of the original devdocs-read-entry – Now auto-filters the entries that match the symbol at point.

Above function just pre-sets the filter.. If the user wants to change the search string, they can still do that.

Using devdocs-lookup

Finally, I like the key-chord.el package. So using that, I bind the ?? key-chord to the modified devdocs-lookup function.

So if I want to look up the docs for walkDirRec on that line in the above example, I just move the point there, and hit ??, and the docs for that will pop up in my browser..

  • No manual launching of the browser.
  • No manual typing of the search string.

Demo

It won’t be fun if I did not end this post without a demo. So here it is —

Figure 1: Click the above image to see the devdocs.io access from Emacs in action (GIF)

Figure 1: Click the above image to see the devdocs.io access from Emacs in action (GIF)

Code

You can find the modified devdocs-lookup code here.


  1. All of the code snippets from Christopher Wellon’s devdocs-lookup that now follow in this post are from this commit. [return]
  2. If you visit that file in Firefox, it will show up in a wonderful formatted form with collapsible drawers, search, etc. [return]
-1:-- Accessing Devdocs from Emacs (Post Kaushal Modi)--L0--C0--May 10, 2018 04:45 PM

Phil Hagelberg: in which interactive development saves the day

When I was writing EXO_encounter 667 in Fennel, I benefited immensely from the ability to do live reloads. Instead of having to restart the whole process, I could run a single key command from my editor and have the game see the new code immediately. This isn't particularly difficult to do in Fennel, but it's not immediately obvious at a glance either.

Before you understand how reloading works in Fennel, you need a little background regarding Lua's module system, since Fennel is just a compiler that emits Lua code. Older versions of Lua had a module function which would declare the whole rest of the file as being part of a specific module and register that with the module system, and all functions that would normally be declared as global within that file would be exported as part of the module instead. But in version 5.1, that system was recognized as redundant: nowadays a module is just a file that returns a table1 with closures and other values in it. This is reflects the relentless simplicity behind the design of Lua; why have modules as their own concept when tables and closures can do just as good a job?

So that's all well and good; you can just write code that uses functions written in other files by just calling dofile on the filename and putting that value in a local. And that works, but every time you use the module from another place it loads a fresh copy, which is wasteful. Enter the require function. It takes a module name which maps to a filename (by searching the entries of package.path) and gives you the value returned by that file, but it also caches subsequent calls. So every time you require a module, you're getting the exact same table2 in the exact same memory location.

Valley near Mt. Saint Helens

We can take a little detour here from Lua land and back into Fennel, because dofile only works on Lua code. Fennel provides its own fennel.dofile function which works just like the built-in one, but on .fnl files instead. But what about require? Well it turns out require is implemented in a pretty clever way that allows us to teach it new tricks. The way require works is that it looks at the package.searchers table, (it's package.loaders on Lua 5.1) which contains a list of searcher functions. It iterates over the list, calling each searcher with the module name. If that returns nil, it indicates that searcher can't find the module and it moves on, but a searcher which can load the module will return a function which allows require to get (and cache) the value for the module in question. So simply by adding fennel.searcher to package.searchers, we can make it so that require works seamlessly on modules whether they are written in Fennel or Lua:

(local fennel (require "fennel"))
(table.insert package.searchers fennel.searcher)

Now this seems somewhat academic; after all, you have a lot of memory; why do you care if modules are duplicated in memory? But using require for modules proved invaluable during the development of my game because it allowed me to do all my local hacking using .fnl files I was constantly editing, but when I prepared a release, I precompiled it all into .lua files and didn't have to change a line of my code to reflect that.

Well that's wonderful, but if require caches the value of each module, doesn't that interfere with live reloading? Indeed it does; simply re-requiring a module has no effect. You can call fennel.dofile to get a copy of the updated module. But that's no help to the existing code which has the old version of the module. What to do?

To understand the solution it's helpful to make a distinction between the identity of the table and the values it contains. The identity of a table is what makes it truly unique; it can be thought of in terms of that table's particular location in memory. When you pass a table to a function, that function has access to the exact same table, and changes made to it inside the function of course are visible to any other function that has access to the table3. The value of a table refers to what it contains; in the case of a module it's usually about what functions are present under what keys. Since the tables are mutable, the value can change over time but the identity cannot. When you call dofile on a module you get a table that might have the same values as last time you called dofile, (if the file on disk hasn't changed) but it will never have the same identity. When you call require you're guaranteed to get the exact same identical table every time.4

With that background maybe you can see now how this might work. All the existing code has access to the original module table. We can't swap out that table for a new one without reloading all the modules that use it, and that can be disruptive. But we can grab that original table, load a fresh copy of its module from disk, then go in and replace its contents with the values from the new one.

(defun fennel-reload-form (module-keyword)
  "Return a string of the code to reload the module-keyword module."
  (format "%s\n" (let [old (require ,module-keyword)
                            _ (tset package.loaded ,module-keyword nil)
                            new (require ,module-keyword)]
                    ;; if the module isnt a table then we can't make
                    ;; changes which affect already-loaded code, but if
                    ;; it is then we should splice new values into the
                    ;; existing table and remove values that are gone.
                    (when (= (type new) :table)
                      (each [k v (pairs new)]
                            (tset old k v))
                      (each [k (pairs old)]
                            ;; the elisp reader is picky about where . can be
                            (when (not (,"." new k))
                              (tset old k nil)))
                      (tset package.loaded ,module-keyword old)))))

The code above looks like Fennel, but it's actually Fennel embedded inside Emacs Lisp code; because they're both just made up of s-expressions, you can write Fennel code as Elisp code and quote it, then send it to the Fennel repl subprocess which is launched with M-x run-lisp. My recent changes to fennel-mode.el allow this to work out of the box, but they could easily be adapted to any other editor that supports communicating with an integrated repl subprocess.

Of course, all this background really isn't necessary; you can just hit reload now and have it work with no fuss. But sometimes it's interesting to understand why it works, and especially I think in this case the design decisions that went into the module system are noteworthy for allowing this kind of thing to be done in a graceful way, so that's worth appreciating and hopefully learning from.


[1] Technically a module can return any value, not just a table. But if you return a non-table, then the reloading features described don't work, because only tables can have their contents replaced while retaining their same object identity.

[2] Yep; this means you can abuse the module system to do terrible things like share application state across other modules. Please resist the temptation.

[3] Oddly enough in some languages this is not true and data structures default to being copied implicitly every time you pass them to a function, which can be very confusing. To muddle things even more, this behavior is referred to as "pass by value" instead of "we make copies of everything for you even when you don't ask". That doesn't happen here.

[4] For a fascinating discussion of the difference between value and identity and how it relates to equality I strongly recommend reading the very insightful Equal Rights for Functional Objects which goes into much more depth on this subject. Notably Lua's (and Fennel's) equality semantics are consistent with its recommendations despite Lua being an imperative language.

-1:-- in which interactive development saves the day (Post Phil Hagelberg)--L0--C0--May 10, 2018 02:11 AM

Raimon Grau: TIL elisp has iterators

So I was reading the Gnu/Emacs mail list and saw this thread where there's some good discussion about destructively modifying a recursive structure (as every structure, right?) in elisp.

All proposed solutions have something worth reading and understanding, but there's one that caught my attention. It uses cl-loop and generators. Yep, like python's ones, but in elisp.

I didn't even know that elisp had this magic in loop (as CL doesn't have it, and you have to make iterators yourself as closures called in `for' clauses.

So yep, the code is (as usual) only elisp and ready to be explored (and be enlightened with).  pcase and CPS used in real world. Also, nice to see in the end of the file where it wrestles itself into elisp itself and into emacs (even it adds syntax highlight for it).

Very good Sunday afternoon read (I just don't understand everything on it yet...)

Thanks to Daniel Colascione again for providing inspiring gems one more time :)
-1:-- TIL elisp has iterators (Post Raimon Grau (noreply@blogger.com))--L0--C0--May 07, 2018 08:36 AM

Phil Hagelberg: in which a game jam is recounted further

This is the second part continuing my previous post about creating the game EXO_encounter 667 using the Fennel programming language and the LÖVE game framework for the Lisp Game Jam 2018; you'll probably want to read the first installment if you haven't already. I wrote about the game design and art, but in this post I'd like to dive into the more technical aspects of the game.

exo encounter terminal

The voting for the game jam just closed, and EXO_encounter 667 came in ranked first! Three out of the top four winners are LÖVE games; one other in Fennel and one in Urn.

Libraries

I pulled in a couple libraries on top of LÖVE to help out in a few areas. First and foremost I would dread to do any work on the Lua runtime without lume, which I like to think of as Lua's "missing standard library". It brings handy things like filter, find, reduce, etc. It's mostly sequence-related functions, but there are a few other handy functions as well like split, a bizarre omission from the standard library, or hotswap which I'll get to below.

The bump.lua library is used for collision detection, and as long as you only need to operate in terms of axis-aligned rectangles, it is very easy to use and gets the job done with no fuss.1 But one of the nicest things about bump is that it's integrated into Simple Tiled Implementation, which handles maps exported from Tiled. On its own the Tiled library just handles drawing them (including their animations and layering), but it can automatically integrate with bump if you set properties on a layer or object to flag it as collidable.

The documentation for the Tiled library unfortunately leaves quite a bit to be desired; it's one of those projects that just dumps a list of all functions with a line or two describing what each one does and considers that "the documentation". Fortunately the source is pretty readable, but figuring out how to handle opening and closing of doors was definitely the roughest spot when it came to 3rd-party libraries. The readme does describe how to implement a custom drawing routine for a layer, which allows us to draw a door differently based on whether it's closed or open. The problem is there's no easy way to do the same thing for the collision detection side of the story.

The Tiled library handles setting up the "world" table from bump by seeding it with all the collidable things from the map. The problem is it doesn't actually use the same tables from the map when adding them to the bump table; it wraps them in bump-specific tables stripping it down to just the fields relevant to collision detection. This is fine until have a door you need to open. Normally you'd do this by calling bump.remove with the door table to make the door no longer take part in collision detection, but bump doesn't know about the door table; it only knows about the wrapper table, which we no longer have access to.

I ended up hacking around this by making the Tiled library save off all the wrapper tables it created, and introducing a new bump_wrap function on the map which would intercept methods on the bump world, accept a regular table and look up the wrapped table and use it instead in the method call. It got the job done quickly, but I couldn't help but feel there should be a better way. I've opened an issue with the Tiled library to see if maybe I missed an undocumented built-in way of doing this. But as far as the coding went, this was really the only hiccup I encountered with any of the libraries I used.

Interactive Development

As a lisp, of course Fennel ships with a REPL (aka interactive console, often mistakenly called an "interpreter") which allows you to enter code and see the results immediately. This is absolutely invaluable for rapid game development. There's a bit of a hiccup though; the REPL reads from standard in, and LÖVE doesn't ship with a method for reading from standard in without blocking. Since Lua doesn't have concurrency, this means reading repl input would block the whole game loop until enter was pressed! LÖVE saves the day here by allowing you to construct "threads" which are really just completely independent Lua virtual machines that can communicate with each other over queues but can't share any data directly. This turns out to be just fine for the repl; one thread can sit and block on standard in, and when it gets input send it over a queue to the main thread which evaluates and sends the response back.

(defn start-repl []
  (let [code (love.filesystem.read "stdio.fnl")
        lua (love.filesystem.newFileData (fennel.compileString code) "io")
        thread (love.thread.newThread lua)
        io-channel (love.thread.newChannel)]
    ;; this thread will send "eval" events for us to consume:
    (: thread :start "eval" io-channel)
    (set love.handlers.eval
         (fn [input]
           (let [(ok val) (pcall fennel.eval input)]
             (: io-channel :push (if ok (view val) val)))))))

As I use Emacs, I've configured fennel-mode to add a key combo for reloading the module for the current buffer. This only works if the current file is in the root directory of the project; it won't work with subdirectories as the module name will be wrong, but it's pretty helpful. It also requires lume be defined as a global variable. (Normally I avoid using globals, but I make two exceptions; one for lume and another for pp as a pretty-print function.) I haven't included this in fennel-mode yet because of these gotchas; maybe if I can find a way to remove them it can be included as part of the mode itself in the future.

Simply run C-u M-x run-lisp to start your game, and use love . as your command. Once that's started, the code below will make C-c C-k reload the current module.

(eval-after-load 'fennel-mode
  '(define-key fennel-mode-map (kbd "C-c C-k")
     (defun pnh-fennel-hotswap ()
       (interactive)
       (comint-send-string
        (inferior-lisp-proc)
        (format "(lume.hotswap \"%s\")\n"
                (substring (file-name-nondirectory (buffer-file-name)) 0 -4))))))

Update: I added first-class support for reloads to fennel-mode, though you will still need the stdin hack described above when using it inside LÖVE.

The other gotcha is that currently an error will crash your whole game. I really wanted to add an error handler which would allow you to resume play after reloading the module that crashed, but I didn't have time to add that. Hopefully I'll have that ready in time for the next jam!

Tutorial

From a usability perspective, one of the most helpful things was adding a tutorial to explain the basic controls and mechanics. The tutorial displays instructions onscreen until the point at which the player carries out those instructions, at which point it moves on to the next instructions. There are various ways you could go about doing this, but I chose to implement it using coroutines, which are Lua's way of offering cooperative multitasking.

(defn tutorial [state world map dt]
  (echo "Press 2 to select rover 2; bring it near the"
        "main probe and press enter to dock.")
  (while (not (. state.rovers 2 :docked?))
    (coroutine.yield))

  (echo "With at least 3 rovers docked, the main" "probe has mobility."
        "" "Now do the same with rover 3.")
  (while (not (. state.rovers 3 :docked?))
    (coroutine.yield))

  (echo "The probe's communications laser can be"
        "activated by holding space. Comma and"
        "period change the aim of the laser.")
  (while (not (or (and state.laser (~= state.selected.theta math.pi))
                  (> (: world :getRect state.selected) 730)
                  (sensor? map "first")))
    (coroutine.yield))

  ...)

The tutorial function runs inside a coroutine started with coroutine.wrap; it echoes the first message and then suspends itself with coroutine.yield which returns control to the caller. On every tick, the love.update function coroutine.resumes it which allows it to check whether the conditions have been fulfilled. If so it can move on to the next instruction; otherwise it just yields back immediately. Of course, it would be possible to do something like this using only closures, but coroutines allow it to be written in a very linear, straightforward way.

exo encounter laser screenshot

Distribution

With LÖVE you get portability across many operating systems; however it does not actually handle creating the executables for each platform. I used an old version of love-release2 to create zip files which include everything you need to run on Windows and Mac OS. This was a huge help; I could run my entire build from my Debian laptop without even touching a Windows machine or a Mac.

For the jam I just published a .love file for other platforms, which requires you to manually install LÖVE yourself. This is a bit of a drag since most package managers don't include the correct version of LÖVE, and even if they did today, in the future they'd upgrade to a different one, so this is one place where relying on the package manager is definitely not going to cut it. Soon after the jam I discovered AppImages which are a way of bundling up all a program's dependencies into a single executable file which should work on any Linux distribution. While I think this is a really terrible idea for a lot of software, for a single-player game that doesn't load any data from untrusted sources, I believe it to be the best option. The love-release tool doesn't currently support creating AppImages, but I am hoping to add support for this. I also didn't get around to automating uploading of builds to itch.io using butler, but I'm hoping to have that working for next time.

Play my game!

Now that the jam is over, I've gotten some great feedback from players that resulted in a nice todo list of items that can be improved. I hope to release a "special edition" in the near future that includes all the things I wasn't able to get to during the jam. But in the mean time, I hope you enjoy EXO_encounter 667!


[1] LÖVE ships with a physics engine built-in, but the API it uses is much more complicated. It's capable of more sophisticated behavior, but unless you really can't work in terms of rectangles, I'd recommend sticking with the much simpler bump.lua.

[2] The love-release project has since been rewritten in Lua instead of being a shell script as it was at the time I downloaded the version I used. I haven't tried the new version but it looks promising.

-1:-- in which a game jam is recounted further (Post Phil Hagelberg)--L0--C0--May 05, 2018 05:05 PM

emacspeak: Emacspeak 48.0 (ServiceDog) Unleashed!

Emacspeak 48.0—ServiceDog—Unleashed!

*For Immediate Release:


San Jose, Calif., (May 04, 2018)


Emacspeak 48.0 (ServiceDog):
Redefining Accessibility In The Age Of User-Aware Interfaces
–Zero cost of Ownership makes priceless software Universally affordable!


Emacspeak Inc (NASDOG: ESPK) — http://github.com/tvraman/emacspeak
— announces the immediate world-wide availability of Emacspeak 48.0
(ServiceDog) — a powerful audio desktop for leveraging today's
evolving Data, Social and Assistant-Oriented Internet cloud.


1 Investors Note:

With several prominent tweeters expanding coverage of #emacspeak,
NASDOG: ESPK has now been consistently trading over the social net at
levels close to that once attained by DogCom high-fliers—and as of
May
2018 is trading at levels close to that achieved by once better known
stocks in the tech sector.


2 What Is It?

Emacspeak is a fully functional audio desktop that provides complete
eyes-free access to all major 32 and 64 bit operating environments. By
seamlessly blending live access to all aspects of the Internet such as
ubiquitous assistance, Web-surfing, blogging, social computing and
electronic messaging into the audio desktop, Emacspeak enables speech
access to local and remote information with a consistent and
well-integrated user interface. A rich suite of task-oriented tools
provides efficient speech-enabled access to the evolving
assistant-oriented social Internet cloud.


3 Major Enhancements:

This version requires emacs-25.1 or later.

  1. Emacs 26 Support 🤻
  1. Locate And Play Media ᭳
  2. Updated EPub Support 🕮
  3. Updated Outloud TTS Server 💬
  4. Espeak-NG support📢
  5. Smart TTS Prompts 🙊
  6. DBus Integration including screenlock via Gnome-ScreenSaver 🚌
  7. MPlayer And Equalizer Presets ≝
  8. VLC front-end 🎹
  9. Updated URL templates 🕷
  10. Updated websearch wizards 🕸


    — And a lot more than will fit this margin. … 🗞



4 Establishing Liberty, Equality And Freedom:

Never a toy system, Emacspeak is voluntarily bundled with all
major Linux distributions. Though designed to be modular,
distributors have freely chosen to bundle the fully integrated
system without any undue pressure—a documented success for
the integrated innovation embodied by Emacspeak. As the system
evolves, both upgrades and downgrades continue to be available at
the same zero-cost to all users. The integrity of the Emacspeak
codebase is ensured by the reliable and secure Linux platform
used to develop and distribute the software.


Extensive studies have shown that thanks to these features, users
consider Emacspeak to be absolutely priceless. Thanks to this
wide-spread user demand, the present version remains priceless
as ever—it is being made available at the same zero-cost as
previous releases.


At the same time, Emacspeak continues to innovate in the area of
eyes-free Assistance and social interaction and carries forward the
well-established Open Source tradition of introducing user interface
features that eventually show up in luser environments.


On this theme, when once challenged by a proponent of a crash-prone
but well-marketed mousetrap with the assertion "Emacs is a system from
the 70's", the creator of Emacspeak evinced surprise at the unusual
candor manifest in the assertion that it would take popular
idiot-proven interfaces until the year 2070 to catch up to where the
Emacspeak audio desktop is today. Industry experts welcomed this
refreshing breath of Courage Certainty and Clarity (CCC) at a time
when users are reeling from the Fear Uncertainty and Doubt (FUD)
unleashed by complex software systems backed by even more convoluted
press releases.


5 Independent Test Results:

Independent test results have proven that unlike some modern (and
not so modern) software, Emacspeak can be safely uninstalled without
adversely affecting the continued performance of the computer. These
same tests also revealed that once uninstalled, the user stopped
functioning altogether. Speaking with Aster Labrador, the creator of
Emacspeak once pointed out that these results re-emphasize the
user-centric design of Emacspeak; “It is the user –and not the
computer– that stops functioning when Emacspeak is uninstalled!”.


5.1 Note from Aster,Bubbles and Tilden:

UnDoctored Videos Inc. is looking for volunteers to star in a
video demonstrating such complete user failure.


6 Obtaining Emacspeak:

Emacspeak can be downloaded from GitHub –see
https://github.com/tvraman/emacspeak you can visit Emacspeak on the
WWW at http://emacspeak.sf.net. You can subscribe to the emacspeak
mailing list — emacspeak@cs.vassar.edu — by sending mail to the
list request address emacspeak-request@cs.vassar.edu. The Emacspeak
Blog
is a good source for news about recent enhancements and how to
use them.


The latest development snapshot of Emacspeak is always available via
Git from GitHub at
Emacspeak GitHub .


7 History:

  • Emacspeak 48.0 (ServiceDog) builds on earlier releases to provide
    continued end-user value.
  • Emacspeak 47.0 (GentleDog) goes the next step in being helpful
    while letting users learn and grow.
  • Emacspeak 46.0 (HelpfulDog) heralds the coming of Smart Assistants.
  • Emacspeak 45.0 (IdealDog) is named in recognition of Emacs'
    excellent integration with various programming language
    environments — thanks to this, Emacspeak is the IDE of choice
    for eyes-free software engineering.
  • Emacspeak 44.0 continues the steady pace of innovation on the
    audio desktop.
  • Emacspeak 43.0 brings even more end-user efficiency by leveraging the
    ability to spatially place multiple audio streams to provide timely
    auditory feedback.
  • Emacspeak 42.0 while moving to GitHub from Google Code continues to
    innovate in the areas of auditory user interfaces and efficient,
    light-weight Internet access.
  • Emacspeak 41.0 continues to improve
    on the desire to provide not just equal, but superior access —
    technology when correctly implemented can significantly enhance the
    human ability.
  • Emacspeak 40.0 goes back to Web basics by enabling
    efficient access to large amounts of readable Web content.
  • Emacspeak 39.0 continues the Emacspeak tradition of increasing the breadth of
    user tasks that are covered without introducing unnecessary
    bloatware.
  • Emacspeak 38.0 is the latest in a series of award-winning
    releases from Emacspeak Inc.
  • Emacspeak 37.0 continues the tradition of
    delivering robust software as reflected by its code-name.
  • Emacspeak 36.0 enhances the audio desktop with many new tools including full
    EPub support — hence the name EPubDog.
  • Emacspeak 35.0 is all about
    teaching a new dog old tricks — and is aptly code-named HeadDog in
    on of our new Press/Analyst contact. emacspeak-34.0 (AKA Bubbles)
    established a new beach-head with respect to rapid task completion in
    an eyes-free environment.
  • Emacspeak-33.0 AKA StarDog brings
    unparalleled cloud access to the audio desktop.
  • Emacspeak 32.0 AKA
    LuckyDog continues to innovate via open technologies for better
    access.
  • Emacspeak 31.0 AKA TweetDog — adds tweeting to the Emacspeak
    desktop.
  • Emacspeak 30.0 AKA SocialDog brings the Social Web to the
    audio desktop—you cant but be social if you speak!
  • Emacspeak 29.0—AKAAbleDog—is a testament to the resilliance and innovation
    embodied by Open Source software—it would not exist without the
    thriving Emacs community that continues to ensure that Emacs remains
    one of the premier user environments despite perhaps also being one of
    the oldest.
  • Emacspeak 28.0—AKA PuppyDog—exemplifies the rapid pace of
    development evinced by Open Source software.
  • Emacspeak 27.0—AKA
    FastDog—is the latest in a sequence of upgrades that make previous
    releases obsolete and downgrades unnecessary.
  • Emacspeak 26—AKA
    LeadDog—continues the tradition of introducing innovative access
    solutions that are unfettered by the constraints inherent in
    traditional adaptive technologies.
  • Emacspeak 25 —AKA ActiveDog
    —re-activates open, unfettered access to online
    information.
  • Emacspeak-Alive —AKA LiveDog —enlivens open, unfettered
    information access with a series of live updates that once again
    demonstrate the power and agility of open source software
    development.
  • Emacspeak 23.0 — AKA Retriever—went the extra mile in
    fetching full access.
  • Emacspeak 22.0 —AKA GuideDog —helps users
    navigate the Web more effectively than ever before.
  • Emacspeak 21.0
    —AKA PlayDog —continued the
    Emacspeak tradition of relying on enhanced
    productivity to liberate users.
  • Emacspeak-20.0 —AKA LeapDog —continues
    the long established GNU/Emacs tradition of integrated innovation to
    create a pleasurable computing environment for eyes-free
    interaction.
  • emacspeak-19.0 –AKA WorkDog– is designed to enhance
    user productivity at work and leisure.
  • Emacspeak-18.0 –code named
    GoodDog– continued the Emacspeak tradition of enhancing user
    productivity and thereby reducing total cost of
    ownership.
  • Emacspeak-17.0 –code named HappyDog– enhances user
    productivity by exploiting today's evolving WWW
    standards.
  • Emacspeak-16.0 –code named CleverDog– the follow-up to
    SmartDog– continued the tradition of working better, faster,
    smarter.
  • Emacspeak-15.0 –code named SmartDog–followed up on TopDog
    as the next in a continuing series of award-winning audio desktop
    releases from Emacspeak Inc.
  • Emacspeak-14.0 –code named TopDog–was

the first release of this millennium.

  • Emacspeak-13.0 –codenamed
    YellowLab– was the closing release of the
    20th. century.
  • Emacspeak-12.0 –code named GoldenDog– began
    leveraging the evolving semantic WWW to provide task-oriented speech
    access to Webformation.
  • Emacspeak-11.0 –code named Aster– went the
    final step in making Linux a zero-cost Internet access solution for
    blind and visually impaired users.
  • Emacspeak-10.0 –(AKA
    Emacspeak-2000) code named WonderDog– continued the tradition of
    award-winning software releases designed to make eyes-free computing a
    productive and pleasurable experience.
  • Emacspeak-9.0 –(AKA
    Emacspeak 99) code named BlackLab– continued to innovate in the areas
    of speech interaction and interactive accessibility.
  • Emacspeak-8.0 –(AKA Emacspeak-98++) code named BlackDog– was a major upgrade to
    the speech output extension to Emacs.
  • Emacspeak-95 (code named Illinois) was released as OpenSource on
    the Internet in May 1995 as the first complete speech interface
    to UNIX workstations. The subsequent release, Emacspeak-96 (code
    named Egypt) made available in May 1996 provided significant
    enhancements to the interface. Emacspeak-97 (Tennessee) went
    further in providing a true audio desktop. Emacspeak-98
    integrated Internetworking into all aspects of the audio desktop
    to provide the first fully interactive speech-enabled WebTop.

8 About Emacspeak:

Originally based at Cornell (NY) —
http://www.cs.cornell.edu/home/raman —home to Auditory User
Interfaces (AUI) on the WWW, Emacspeak is now maintained on GitHub
https://github.com/tvraman/emacspeak. The system is mirrored
world-wide by an international network of software archives and
bundled voluntarily with all major Linux distributions. On Monday,
April 12, 1999, Emacspeak became part of the Smithsonian's Permanent
Research Collection
on Information Technology at the Smithsonian's
National Museum of American History.


The Emacspeak mailing list is archived at Vassar –the home of the
Emacspeak mailing list– thanks to Greg Priest-Dorman, and provides a
valuable knowledge base for new users.


9 Press/Analyst Contact: Tilden Labrador

Going forward, Tilden acknowledges his exclusive monopoly on
setting the direction of the Emacspeak Audio Desktop, and
promises to exercise this freedom to innovate and her resulting
power responsibly (as before) in the interest of all dogs.


*About This Release:



Windows-Free (WF) is a favorite battle-cry of The League Against
Forced Fenestration (LAFF). –see
http://www.usdoj.gov/atr/cases/f3800/msjudgex.htm for details on
the ill-effects of Forced Fenestration.


CopyWrite )C( Aster, Hubbell and Tilden Labrador. All Writes Reserved.
HeadDog (DM), LiveDog (DM), GoldenDog (DM), BlackDog (DM) etc., are Registered
Dogmarks of Aster, Hubbell and Tilden Labrador. All other dogs belong to
their respective owners.

-1:-- Emacspeak 48.0 (ServiceDog) Unleashed! (Post T. V. Raman (noreply@blogger.com))--L0--C0--May 03, 2018 08:38 PM

Pragmatic Emacs: Write code comments in org-mode with poporg

I recently discovered a nice package called poporg that allows you to pop open a temporary org-mode window when you are editing code comments so you can write them using nice org-mode formatting. When you are done, you close the window and go back to your code with the newly edited comment. This is a bit antithetical to embedding source code in org-mode files to document the code that way, but in fact I find I do a bit of both and still like to have comments in my code, so I really appreciate being able to format them nicely.

You can install the package by adding the following to your emacs config file

(use-package poporg
      :bind (("C-c /" . poporg-dwim)))

With this, hitting C-c / in your source code in or near a comment will open the org-mode window and the same key combo will close it again.

-1:-- Write code comments in org-mode with poporg (Post Ben Maughan)--L0--C0--May 01, 2018 08:10 PM

Raimon Grau: bootstrap your emacs lisp learning

So every now and then there's someone asking in /r/emacs about learning elisp. And I always answer with the same solution that I'm now posting here to avoid copypasting the whole thing every time.

In an attempt to give bootstrapped solution, t-shirt sized, and that is both a fish,  a fishing lesson and a fishing rod, this is what I came with.
 
  • Type c-h k c-h k and read.
  • Type c-h k c-h f and read.
  • With the previous tools, try to understand the following:
    (defun foo ()
    (interactive)
    (save-excursion
    (beginning-of-line)
    (insert ";")))

    (global-set-key (kbd "C-c ;") 'foo)
  • Select the whole snippet and type m-x eval-region.
  • Type c-c ; .
  • Try to understand what just happened.
-1:-- bootstrap your emacs lisp learning (Post Raimon Grau (noreply@blogger.com))--L0--C0--April 30, 2018 01:40 PM

Marcin Borkowski: org-mru-clock

After writing about counsel-org-clock, I got an email from Kevin Brubeck Unhammer, the author of another package facilitating clocking recent things: org-mru-clock (where “mru” stands for “most recently used” items).
-1:-- org-mru-clock (Post)--L0--C0--April 28, 2018 06:24 AM

Timo Geusch: Another way to use Emacs to convert DOS/Unix line endings

I’ve previously blogged about using Emacs to convert line endings and use it as an alternative to the dos2unix/unix2dos tools. Using set-buffer-file-coding-system works well and has been my go-to conversion method. That said, there is another way to do the Read More

The post Another way to use Emacs to convert DOS/Unix line endings appeared first on The Lone C++ Coder's Blog.

-1:-- Another way to use Emacs to convert DOS/Unix line endings (Post Timo Geusch)--L0--C0--April 24, 2018 01:04 PM

Ben Simon: A Little Scheme Setup and Development on the Galaxy S9 Plus

Today's goal was to setup a Scheme programming environment on my new Galaxy S9 Plus phone. I planned to use the same tools I'd used on my LG G6:

  • Termux - provides a Unixy environment, doesn't require root and is just plain awesome
  • tinyscheme - provides a lightweight, yet powerful, scheme environment. For bonus points, it's trivial to install within Termux. Just type: pkg install tinyscheme
  • emacs, vim, git - these tools run well under Termux and are installed using the pkg install command. I use the same emacs config on my phone as I do my desktop.
  • External Keyboard Helper - remaps keys on my hardware keyboard. Specifically, I use it to map Caps Lock to Escape. I've found that this keybinding is essential to making bash, emacs and vim work in a sane way.

With the tools installed, I was able to open up a new .scm file in emacs, type C-u M-x run-scheme tinyscheme and I was off and running with an interactive Scheme development environment.

While the tools installed without issue, I couldn't consider the environment setup until I wrote some code. So off I went to ProgrammingPraxis.com to find an interesting exercise to tackle.

Sticking with the theme of rendering text in unique ways, I decided to complete the Seven-Segment Devices exercise. A Seven-Segment Device is a primitive display that renders numbers and text using seven different lit up bars. Think old school calculators. Here are the seven different segments you have to work with:

 --2--
|     |
3     4
|     |
 --1--
|     |
5     6
|     |
 --0--

Below you'll find my solution that turns any integer into text that's rendered using this seven-segment style. The exercise called for storing the data as a bit-array. I took an alternative approach, and mapped every digit in the number to a list of enabled segments. For example, 0 maps to (2 4 6 0 5 3). This is more verbose than packing the data into bits, but I was more interested in rendering the text than storing a compact representation of it.

Here are some screenshots of the code running, and the code itself is below. It felt good to be coding on my new phone, and it sure is remarkable how well all these tools Just Work.

;; https://programmingpraxis.com/2018/02/27/seven-segment-devices/


(define (show . args)
  (display args)
  (newline))

(define (range lower upper)
  (if (< lower upper)
    (cons lower (range (+ 1 lower) upper))
    '()))

(define *digit-width* 10)
(define *digit-height* 11)
(define *digit-sep* 4)

(define *digit-map*
  '((0 . (2 4 6 0 5 3))
    (1 . (4 6))
    (2 . (2 4 1 5 0))
    (3 . (2 4 1 6 0))
    (4 . (3 1 4 6))
    (5 . (2 3 1 6 0))
    (6 . (3 5 0 6 1))
    (7 . (2 4 6))
    (8 . (0 1 2 3 4 5 6))
    (9 . (2 4 1 3 6))))

(define (integer->bars val)
  (if (= val 0)
    (cdr (assoc 0 *digit-map*))
    (let loop ((val val) (bars '()))
      (cond ((<= val 0) 
             bars)
            (else
             (let ((d (modulo val 10)))
               (loop
                (inexact->exact (floor (/ val 10)))
                (cons (cdr (assoc d *digit-map*)) bars))))))))
  

(define (draw-sep)
  (for-each (lambda (i)
              (display " "))
            (range 0 *digit-sep*)))

(define (draw first mid last)
  (display first)
  (for-each (lambda (i)
              (display mid))
            (range 0 (- *digit-width* 2)))
  (display last))

(define (draw-row digits row)
  (define mid (/ (- *digit-height* 1) 2))
  (define top? (= row 0))
  (define bottom? (= row (- *digit-height* 1)))
  (define mid? (= row mid))
  (define upper? (and (not top?) (< row mid)))
  (define lower? (and (not bottom?) (> row mid)))
  (for-each (lambda (bars)
              (define (has? x) (member x bars))
              (cond ((and top? (has? 2))
                     (draw "+" "-" "+"))
                    ((and bottom? (has? 0))
                     (draw "+" "-" "+"))
                    ((and mid? (has? 1))
                     (draw "+" "-" "+"))
                    ((and upper? (has? 3) (has? 4))
                     (draw "|" " " "|"))
                    ((and upper? (has? 3))
                     (draw "|" " " " "))
                    ((and upper? (has? 4))
                     (draw " " " " "|"))
                    ((and lower? (has? 5) (has? 6))
                     (draw "|"   " "   "|"))
                    ((and lower? (has? 5))
                     (draw "|" " " " "))
                    ((and lower? (has? 6))
                     (draw " " " " "|"))
                    (else
                     (draw " " " " " ")))
              (draw-sep))
            digits)
  (newline))


(define (draw-integer x)
  (define s (integer->bars x))
  (newline)
  (for-each (lambda (row)
              (draw-row s row))
            (range 0 *digit-height*))
  x)
-1:-- A Little Scheme Setup and Development on the Galaxy S9 Plus (Post Ben Simon (noreply@blogger.com))--L0--C0--April 24, 2018 03:11 AM

Timo Geusch: Emacs 26.1-RC1 on the Windows Subsystem for Linux

As posted in a few places, Emacs 26.1-RC1 has been released. Following up my previous experiments with running Emacs on the Windows Subsystem for Linux, I naturally had to see how the latest version would work out. For that, I Read More

The post Emacs 26.1-RC1 on the Windows Subsystem for Linux appeared first on The Lone C++ Coder's Blog.

-1:-- Emacs 26.1-RC1 on the Windows Subsystem for Linux (Post Timo Geusch)--L0--C0--April 19, 2018 10:07 PM

Rubén Berenguel: More emacs configuration tweaks (multiple-cursor on click, minimap, code folding, ensime eval overlays)

At Affectv we use a wide range of editors: Sublime, Atom, Emacs, Pycharm, IntelliJ... Actually only two people use the same editor! As such, from time to time I see things in other people's editors that I would like to have as well. So, yesterday I decided to improve on some configuration settings on Spacemacs.

Click for multiple-cursors

I saw this on Jordi's Sublime, and it is much more comfortable than using more-like-this or similar helper functions, even if I need to use the trackpad to do so. After all, a multi-cursor edit (proper edit, not as a substitute for a macro) is rare enough that I can tolerate leaving the home row. Easy enough to configure thanks to Magnar Sveen.

(global-unset-key (kbd "M-<down-mouse-1>"))
(global-set-key (kbd "M-<mouse-1>") 'mc/add-cursor-on-click)

Minimap

Also from Sublime, I used to have this on my old emacs setup. As simple as adding minimap to the list of additional packages and configuring its property group. See animation below.

dotspacemacs-additional-packages '(helm-dash key-chord pig-mode mmm-mode minimap origami ansible)

Folding

I have always loved how clean vim's folding works, and how Sublime has this nice folding. Then I found origami-mode and my emacs-life was complete. I tweaked a little the folding functions so that minimap was updated on fold (for some reason it is not, I guess minimap is tied to the "modified" hook or similar). I bound z and Z (and A-z which maps to æ in Colemak) to the basic fold operations.

(eval-after-load 'origami
'(progn
(defun rb-show-only (buffer point)
(interactive (list (current-buffer) (point)))
(progn (origami-show-only-node buffer point)
(minimap-new-minimap)))

(defun rb-toggle-rec (buffer point)
(interactive (list (current-buffer) (point)))
(progn (origami-recursively-toggle-node buffer point)
(minimap-new-minimap)))

(define-key evil-normal-state-map "æ" 'rb-show-only)
(define-key evil-normal-state-map "Z" 'origami-toggle-node)
(define-key evil-visual-state-map "Z" 'origami-toggle-node)
(define-key evil-insert-state-map "C-Z" 'origami-toggle-node)
(define-key evil-normal-state-map "z" 'rb-toggle-rec)
(define-key evil-visual-state-map "z" 'rb-toggle-rec)
(define-key evil-insert-state-map "C-z" 'rb-toggle-rec)
)))


For some reason just advising the functions with after didn't work, this is not great but does work. I left the Z bindings as they are, since I have not used them yet, and will probably delete them if I keep not using them.



Note (2018): I moved to use yafolding (specifically yafolding-toggle-all and yafolding-toggle-element) at the end of 2017. Although origami is very good, yafolding works seamlessly under more languages. I bound Return in normal mode to toggle element and C-return to toggle all, as well as unbounding all the default bindings.

Execution overlays in Ensime (Scala)

I saw this for Cider in the emacs church meeting from August, and heard @fommil (I think it was him) mention that it was coming to ensime. And indeed it was. And it's easy enough to use C-c C-v C-r (thing of it as extended command, eval, region to remember), given an open inferior Scala interpreter. Symbol prettify does not apply to overlays, so you need to customise the arrow used therein.




-1:-- More emacs configuration tweaks (multiple-cursor on click, minimap, code folding, ensime eval overlays) (Post Rubén Berenguel (noreply@blogger.com))--L0--C0--April 14, 2018 04:13 PM

Rubén Berenguel: SyncTeX and pdf-view-mode for emacs

Or, destiny is cruel
Back in the days of yore, when I was switching between my Windows machine and a Linux machine, I remember having SyncTeX active in my Windows machine. It was a wonderful experience: SyncTeX lets you click anywhere on a generated file from LaTeX and gets back to your editor, to the place generating where you clicked. This was extremely useful, specially later on when you need to adjust many formulas to fit and you need a bit of back-and-forth-ing.

Then I got a Mac, and since Preview is so handy I slowly forgot about SyncTeX. Time went on, and I merrily kept on editing LaTeX files as usual. I even managed to deliver my PhD dissertation a couple weeks ago, the formal speech will be in a month or two (come at your own risk). AucTeX’s preview saved most of the days, so I slowly even forgot SyncTeX existed. Shame on me indeed.

The other day I got an annotated PDF from one of my advisors, and I just couldn’t open the annotations. I tried all programs I had for Mac, and no luck: annotations weren’t showing, just saw the icons. Surveying for some command-line tool to extract annotations (just in case) I found pdf-tools, a replacement for DocView based on Poppler. It had the awesome ability of actually displaying annotations, with it it was pretty clear the annotations were broken in that PDF. I got a new set of PDFs from my advisor with the annotations in place, though. While waiting for it to arrive…

I saw SyncTeX was an option of pdf-tools. I had been using that, hadn’t I? So, I activated SyncTeX in AucTeX (it is TeX-source-correlate-method, see here) and indeed: I could have two frames, one with the actual LaTeX sources and the other with a PDF, and go from one to the other. Even hyperreferences in PDF work! See (well, click on the full-screen mode to see it larger or you won't see anything)!



Getting pdf-tools to work wasn’t incredibly tricky (given the hoops you need for some packages, sometimes). Just

brew install pdf-tools

and after reading

brew info pdf-tools

I was told to run

  emacs -Q --batch --eval "(package-install-file 
\"/usr/local/Cellar/pdf-tools/0.60/pdf-tools-0.60.tar\")"

and this does the trick (well, change emacs for your actual emacs, which likely is /Applications/Emacs.app/Contents/MacOS/Emacs) You’ll also need to add to your .emacs file (or temporarily in your *scratch* buffer)


(setenv "PKG_CONFIG_PATH" (concat "/usr/local/Cellar/zlib/1.2.8/lib/pkgconfig" ":"

"/usr/local/lib/pkgconfig:/opt/X11/lib/pkgconfig"))


(getenv "PKG_CONFIG_PATH")


and run

(pdf-tools-install)

as advised in the package’s README. And that's it, open a PDF and activate pdf-view-mode to check everything is in place. Well worth it!
-1:-- SyncTeX and pdf-view-mode for emacs (Post Rubén Berenguel (noreply@blogger.com))--L0--C0--April 08, 2018 06:22 PM

Chen Bin (redguardtoo): Use Imenu to list comments in current buffer

evil-nerd-commenter v3.2.0 has a new function evilnc-imenu-create-index-function.

Imenu could use this function to list all comments in current file.

Usage:

(require 'counsel)
(defun counsel-imenu-comments ()
  "Imenu display comments."
  (interactive)
  (let* ((imenu-create-index-function 'evilnc-imenu-create-index-function))
    (counsel-imenu)))

Screen cast:

counsel-imenu-comments.gif

-1:-- Use Imenu to list comments in current buffer (Post Chen Bin)--L0--C0--April 08, 2018 02:12 AM

Alex Bennée: Working with dired

I’ve been making a lot more use of dired recently. One use case is copying files from my remote server to my home machine. Doing this directly from dired, even with the power of tramp, is a little too time consuming and potentially locks up your session for large files. While browsing reddit r/emacs I found a reference to this post that spurred me to look at spawning rsync from dired some more.

Unfortunately the solution is currently sitting in a pull-request to what looks like an orphaned package. I also ran into some other problems with the handling of where rsync needs to be run from so rather than unpicking some unfamiliar code I decided to re-implement everything in my own package.

I’ve still got some debugging to do to get it to cleanly handle multiple sessions as well as a more detailed mode-line status. Once I’m happy I’ll tag a 0.1 and get it submitted to MELPA.

While getting more familiar with dired I also came up with this little helper:

(defun my-dired-frame (directory)
  "Open up a dired frame which closes on exit."
  (interactive)
  (switch-to-buffer (dired directory))
  (local-set-key
   (kbd "C-x C-c")
   (lambda ()
     (interactive)
     (kill-this-buffer)
     (save-buffers-kill-terminal 't))))

Which is paired with a simple alias in my shell setup:

alias dired="emacsclient -a '' -t -e '(my-dired-frame default-directory)'"

This works really nicely for popping up a dired frame in your terminal window and cleaning itself up when you exit.

-1:-- Working with dired (Post Alex)--L0--C0--April 07, 2018 10:12 AM

Alexander Gromnitsky: Emacs, xkb & xxkb

The song that never ends.

Every kid has its own scheme. Through the years I've picked out mine.

The prerequisites:

  1. You have tuned up xkb and know what key is your X layout switcher. (In the past I've used left Ctrl to toggle en/ru, but recently switched to capslock.)
  2. You're using xxkb as your X Window indicator of the current layout. Not from Gnome, not from KDE, etc.

We want: use capslock for emacs toggle-input-method too. We don't want to modify xorg system files or hack part of emacs written in C.

The problem is that emacs (under xorg-7.5.1, for example) ignores ISO_Next_Group completely. You cannot bind anything to the key that emits that event.

But there is a clever trick I've googled. You translate ISO_Next_Group to some unused key in emacs, for example Super-A. (Super == left win key on my machine.):

 (setf (gethash #xfe08 x-keysym-table) (aref (kbd "s-a") 0)) 

#xfe08 is ISO_Next_Group code. Then bind Super-A to toggle emacs keyboard layout:

 (global-set-key (kbd "s-a") '(lambda () (toggle-input-method))) 

OK. Next we need somehow disable system X switcher for all emacs windows. For xxkb this is done by adding this to its configuration file:

 XXkb.app_list.wm_class_class.alt_group1: Emacs 

And that's all.

As a bonus, I've added another very handy indicator of the current input method in emacs: cursor color.

;; default cursor color for all X frames
(add-to-list 'default-frame-alist '(cursor-color . "black"))

;; toggle the input method and the cursor color in one place
(global-set-key (kbd "s-a") '(lambda ()
                             (interactive)
                             (toggle-input-method)
                             (if window-system
                                 (set-cursor-color
                                  (if current-input-method
                                      "red3"
                                    "black")))
                             ))
-1:-- Emacs, xkb &amp; xxkb (Post ag (noreply@blogger.com))--L0--C0--April 02, 2018 01:25 PM

Alexander Gromnitsky: 'indent-region' in the Emacs batch mode

I had an old project of mine that was using an archaic tab-based indentation w/ the assumed tab width of 4. The mere opening of its files in editors other than Emacs was causing such pain that I decided to do what you should rarely do--reindent the whole project.

(A side note: if you're setting the tab width to a value other than 8 & simultaneously is using tabs for indentation, you're a bad person.)

The project had ~40 files. Manually using Emacs indent-region for each file would have been too wearisome. Then I remembered that Emacs has the batch mode.

A quick googling gave me a recipe akin to:

$ emacs -Q -batch FILE --eval '(indent-region (point-min) (point-max))' \
  -f save-buffer

It would have worked but it had several drawbacks as a general solution, for it:

  1. modifies a file in-place;
  2. can't read from the stdin;
  3. doesn't work w/ multiple files forcing a user to use xargs(1) of smthg;
  4. if FILE doesn't exist, Emacs quietly creates an empty file, whereas a user probably expects to see an error message.

Whereas an ideal little script should:

  1. never overwrite the source file;
  2. read the input from the stdin or from its arguments;
  3. spit out the result to the stdout;
  4. handle the missing input reasonably well.

Everything else should be accommodated by the shell, including the item #2 from the 'drawbacks' list above.

Using Emacs for such a task is tempting, for it supports a big number of programming modes, giving us the ability to employ the editor as a universal reindenting tool, for example, in makefiles or Git hooks.

Writing to the stdout

We can slightly modify the above recipe to:

$ emacs -Q -batch FILE --eval '(indent-region (point-min) (point-max))' \
  --eval '(princ (buffer-string))'

In -f stead we are using --eval twice. buffer-string function does exactly what it says: returns the contents of the current buffer as a string.

$ cat 1.html
<p>
hey, <i>
what's up</i>?
</p>

$ emacs -Q -batch 1.html --eval '(indent-region (point-min) (point-max))' \
  --eval '(princ (buffer-string))'
Indenting region...
Indenting region...done
<p>
  hey, <i>
    what's up</i>?
</p>

The "Indenting region" lines come from Emacs message function (which the progress reporter uses). In the batch mode message prints the lines to the stderr.

The solution also address the item #3 from the "drawbacks" list--Emacs doesn't create an empty file on the invalid input, although it doesn't indicate the error properly, i.e., it fails w/ the exit code 0.

Processing all files at once

If you know what are you doing & the files you're going to process are under Git, overwriting the source it not a big deal. Perhaps for a quick hack this script will do:

:; exec emacs -Q --script "$0" -- "$@" # -*- emacs-lisp -*-

(defun indent(file)
  (set-buffer "*scratch*")
  (if (not (file-directory-p file))
      (when (and (file-exists-p file) (file-writable-p file))
        (message "Indenting `%s`" file)

        (find-file file)
        (indent-region (point-min) (point-max))
        (save-buffer))))

(setq args (cdr argv))                  ; rm --
(dolist (val args)
  (indent val))

If you put the src above into a file named emacs-indent-overwrite & add executable bits to it, then the shell thinks it's a usual sh-script that doesn't have a shebang line. A colon in sh is a noop, but on stumbling upon exec the shell replaces itself with the command

emacs -Q --script emacs-indent-overwrite -- arg1 arg2 ...

When Emacs reads the script, it doesn't expect it to be a sh one, but hopefully the file masks itself as a true Elisp, for in Elisp a symbol whose name starts with a colon is called a keyword symbol. : is a symbol w/o a name (a more usual constant would be :omg) that passes the check because it satisfies keywordp predicate:

ELISP> (keywordp :omg)
t
ELISP> (keywordp :)
t

Everything else in the file is usual Elisp code. Here's how it works:

$ emacs-indent-overwrite src/*
Indenting ‘src/1.html‘
Indenting region...
Indenting region...done
Indenting ‘src/2.html‘
Indenting region...
Indenting region...done

The only thing worth mentioning about the script is why indent procedure has (set-buffer "*scratch*") call. It's an easy way to switch the current directory to the directory from which the script was started. This is required, for find-file modifies the default directory of the current buffer (via modifying a buffer-local default-directory variable). The other way is to modify args w/ smthg like

(setq args (mapcar 'expand-file-name args))

& just pass the file argument as a full path.

Reading from the stdin

There is next to none info about the stdin in the Emacs Elisp manual. The section about minibuffers hints us that for reading from the standard input in the batch mode we ought to seek out for functions that normally read kbd input from the minibuffer.

One of such functions is read-from-minibuffer.

$ cat read-from-minibuffer
:; exec emacs -Q --script "$0"
(message "-%s-" (read-from-minibuffer ""))

$ printf hello | ./read-from-minibuffer
-hello-
$ printf 'hello\nworld\n' | ./read-from-minibuffer
-hello-

The function only read the input up to the 1st newline which it also impertinently ate. This leaves us w/ a dilemma: if we read multiple lines in a loop, should we unequivocally assume that the last line contained the newline?

read-from-minibuffer has another peculiarity: on receiving the EOF character it raises an unhelpful error:

$ ./read-from-minibuffer
^D
Error reading from stdin
$ echo $?
255

The simplest Elisp cat program must watch out for that:

$ ./cat.el < cat.el
:; exec emacs -Q --script "$0"
(while (setq line (ignore-errors (read-from-minibuffer "")))
  (princ (concat line "\n")))

Next, if we read the stdin & print the result to the stdout, our "ideal" reindent script cannot rely on find-file anymore, unless we save the input in a tmp file. Or we can leave out "heavy" find-file altogether & just create a temp buffer & copy the input there for the further processing. The latter implies we must manually set the proper major mode for the buffer, otherwise indent-region won't do anything good.

set-auto-mode function does the major mode auto-detection. One of the 1st hints it looks for is the file extension, but the stdin has none. We can ask the user to provide one in the arguments of the script.

:; exec emacs -Q --script "$0" -- "$@" # -*- emacs-lisp -*-

(defun indent(mode text)
  (with-temp-buffer
    (set-visited-file-name mode)
    (insert text)
    (set-auto-mode t)
    (message "`%s` major mode: %s" mode major-mode)

    (indent-region (point-min) (point-max))
    (buffer-string)))

(defun read-file(file)
  (with-temp-buffer
    (insert-file-contents file)
    (buffer-string)))

(defun read-stdin()
  (let (line lines)
    (while (setq line (ignore-errors (read-from-minibuffer "")))
      (push line lines))
    (push "" lines)
    (mapconcat 'identity (reverse lines) "\n")
    ))

 

(setq args (cdr argv))                  ; rm --

(setq mode (car args))
(if (equal "-" mode)
    (progn
      (setq mode (nth 1 args))
      (if (not mode) (error "No filename hint argument, like .js"))
      (setq text (read-stdin)))
  (setq text (read-file mode)))

(princ (indent mode text))

If the 1st argument to the script is '-', we look for the hint in the next argument & then start reading the stdin. It the 1st arg isn't '-' we assume it's a file that we insert into a temp buffer & return its contents as a string.

The

(mapconcat 'identity (reverse lines) "\n")

line in read-stdin procedure is an equivalent of

['world', 'hello'].reverse().join("\n")

in JavaScript.

Some examples:

$ printf "<p>\nhey, what's up?\n</p>" | ./emacs-indent - .html
‘.html‘ major mode: html-mode
Indenting region...
Indenting region...done
<p>
  hey, what's up?
</p>

$ printf "<p>\nhey, what's up?\n</p>" | ./emacs-indent - .txt
‘.txt‘ major mode: text-mode
Indenting region...
Indenting region...done
<p>
hey, what's up?
</p>

$ ./emacs-indent src/2.html 2>/dev/null
<p>
  not much
</p>
-1:-- 'indent-region' in the Emacs batch mode (Post ag (noreply@blogger.com))--L0--C0--April 02, 2018 01:14 PM

Bryan Murdock: Fixing xref-find-references

I was annoyed that xref-find-references searched more than the files in the TAGS file (it seems to search everything below the top level directory of the TAGS file?) so I went looking for a fix. I found that apparently my emacs knowledge is out of date (the fact that it's xref now and not tags was my first clue). I couldn't find any way to customize xref-find-references. Instead I found people referring to project, ivy, helm, confirm, The Silver Searcher (ag), ripgrep (rg), and dumb jump. I didn't go all the way and get into project or ivy or any of the others, but I did download ag and rg and tried them from a command-line outside of emacs and saw exactly what I was expecting xref-find-references to do. I figured all I needed was to replace xref-find-references with one of those. I got ag.el installed and working before any of the ripgrep packages (there's both rg.el and ripgrep.el) and then struggled to remap M-? to call ag-project instead of xref-find-references. The thing that finally worked was remapping commands. Here's the magic:

(define-key global-map [remap xref-find-references] 'ag-project)

And actually, to work completely like xref-find-references I added one option to the ag command, --word-regexp, like so (oh, I also removed --stats which is there by default in ag.el):

(setq ag-arguments (list "--word-regexp" "--smart-case"))

Much better. Are all those other packages worth digging into? I'm not particularly unhappy with ido.
-1:-- Fixing xref-find-references (Post Bryan (noreply@blogger.com))--L0--C0--March 30, 2018 05:49 PM

Jonas Bernoulli: Magit 2.12 released

I am excited to announce the release of Magit version 2.12, consisting of 610 commits since the last feature release six months ago.
-1:-- Magit 2.12 released (Post)--L0--C0--March 29, 2018 07:30 PM

Alex Bennée: Solving the HKG18 puzzle with org-mode

One of the traditions I like about Linaro’s Connect event is the conference puzzle. Usually set by Dave Piggot they provide a challenge to your jet lagged brain. Full disclosure: I did not complete the puzzle in time. In fact when Dave explained it I realised the answer had been staring me in the face. However I thought a successful walk through would make for a more entertaining read 😉

First the Puzzle:

Take the clues below and solve them. Once solved, figure out what the hex numbers mean and then you should be able to associate each of the clue solutions with their respective hex numbers.

Clue Hex Number
Lava Ale Code 1114DBA
Be Google Roe 114F6BE
Natural Gin 114F72A
Pope Charger 121EE50
Dolt And Hunk 12264BC
Monk Hops Net 122D9D9
Is Enriched Tin 123C1EF
Bran Hearing Kin 1245D6E
Enter Slim Beer 127B78E
Herbal Cabbages 1282FDD
Jan Venom Hon Nun 12853C5
A Cherry Skull 1287B3C
Each Noun Lands 1298F0B
Wave Zone Kits 12A024C
Avid Null Sorts 12A5190
Handcars All Trim 12C76DC

Clues

It looks like all the clues are anagrams. I was lazy and just used the first online anagram solver that Google pointed me at. However we can automate this by combining org-mode with Python and the excellent Beautiful Soup library.

from bs4 import BeautifulSoup
import requests
import re

# ask internet to solve the puzzle
url="http://anagram-solver.net/%s" % (anagram.replace(" ", "%20"))
page=requests.get(url)

# fish out the answers
soup=BeautifulSoup(page.text)
answers=soup.find("ul", class_="answers")
for li in answers.find_all("li"):
    result = li.text
    # filter out non computer related or poor results
    if result in ["Elmer Berstein", "Tim-Berners Lee", "Babbage Charles", "Calude Shannon"]:
        continue
    # filter out non proper names
    if re.search("[a-z] [A-Z]", result):
        break

return result

So with :var anagram=clues[2,0] we get

Ada Lovelace

I admit the “if result in []” is a bit of hack.

Hex Numbers

The hex numbers could be anything. But lets first start by converting to something else.

Hex Prompt Number
1114DBA 17911226
114F6BE 18151102
114F72A 18151210
121EE50 19000912
12264BC 19031228
122D9D9 19061209
123C1EF 19120623
1245D6E 19160430
127B78E 19380110
1282FDD 19410909
12853C5 19420101
1287B3C 19430204
1298F0B 19500811
12A024C 19530316
12A5190 19550608
12C76DC 19691228

The #+TBLFM: is $1='(identity remote(clues,@@#$2))::$2='(string-to-number $1 16)

This is where I went down a blind alley. The fact all they all had the top bit set made me think that Dave was giving a hint to the purpose of the hex number in the way many cryptic crosswords do (I know he is a fan of these). However the more obvious answer is that everyone in the list was born in the last millennium.

Looking up Birth Dates

Now I could go through all the names by hand and look up their birth dates but as we are automating things perhaps we can use computers for what they are good at. Unfortunately there isn’t a simple web-api for looking up this stuff. However there is a project called DBpedia which takes Wikipedia’s data and attempts to make it semantically useful. We can query this using a semantic query language called SparQL. If only I could call it from Emacs…

PREFIX dbr: <http://dbpedia.org/resource/>
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX dbp: <http://dbpedia.org/property/>

select ?birthDate where {
  { dbr:$name dbo:birthDate ?birthDate }
  UNION
  { dbr:$name dbp:birthDate ?birthDate }
}

So calling with :var name="Ada_Lovelace" we get

"birthDate"
1815-12-10
1815-12-10

Of course it shouldn’t be a surprise this exists. And in what I hope is a growing trend sparql-mode supports org-mode out of the box. The $name in the snippet is expanded from the passed in variables to the function. This makes it a general purpose lookup function we can use for all our names.

There are a couple of wrinkles. We need to format the name we are looking up with underscores to make a valid URL. Also the output spits out a header and possible multiple birth dates. We can solve this with a little wrapper function. It also introduces some rate limiting so we don’t smash DBpedia’s public SPARQL endpoint.

;; rate limit
(sleep-for 1)
;; do the query
(let* ((str (s-replace-all '((" " . "_") ("Von" . "von")) name))
       (ret (eval
             (car
              (read-from-string
               (format "(org-sbe get-dob (name $\"%s\"))" str))))))
  (string-to-number (replace-regexp-in-string "-" "" (car (cdr (s-lines ret))))))

Calling with :var name="Ada Lovelace" we get

18151210

Full Solution

So now we know what we are doing we need to solve all the puzzles and lookup the data. Fortunately org-mode’s tables are fully functional spreadsheets except they are not limited to simple transformations. Each formula can be a fully realised bit of elisp, calling other source blocks as needed.

Clue Solution DOB
Herbal Cabbages Charles Babbage 17911226
Be Google Roe George Boole 18151102
Lava Ale Code Ada Lovelace 18151210
A Cherry Skull Haskell Curry 19000912
Jan Venom Hon Nun John Von Neumann 19031228
Pope Charger Grace Hopper 19061209
Natural Gin Alan Turing 19120623
Each Noun Lands Claude Shannon 19160430
Dolt And Hunk Donald Knuth 19380110
Is Enriched Tin Dennis Ritchie 19410909
Bran Hearing Kin Brian Kernighan 19420101
Monk Hops Net Ken Thompson 19430204
Wave Zone Kits Steve Wozniak 19500811
Handcars All Trim Richard Stallman 19530316
Enter Slim Beer Tim Berners-Lee 19550608
Avid Null Sorts Linus Torvalds 19691228

The #+TBLFM: is $1='(identity remote(clues,@@#$1))::$2='(org-sbe solve-anagram (anagram $$1))::$3='(org-sbe frob-dob (name $$2))

The hex numbers are helpfully sorted so as long as we sort the clues table by the looked up date of birth using M-x org-table-sort-lines we are good to go.

You can find the full blog post in raw form here.

-1:-- Solving the HKG18 puzzle with org-mode (Post Alex)--L0--C0--March 26, 2018 10:19 AM

Flickr tag 'emacs': url

kabads posted a photo:

url

-1:-- url (Post kabads (nobody@flickr.com))--L0--C0--March 23, 2018 06:07 PM

emacsninja: Bridging the Ancient and the Modern

I tried out some new social networks lately. Mastodon I quite like (it’s like what I’ve wanted Twitter to be), Discord, not so sure. So, if you’ve wondered about my reduced presence on IRC, that’s why.

Writing an IRC bot is one of the classic programming exercises that can be done in pretty much every programming language offering you some way to open TCP sockets and manipulate strings. I started doing one in Emacs Lisp long time ago, although not from scratch (rather by leveraging an existing IRC client) and wondered whether there is anything to learn from doing the equivalent with a “modern” IM platform like Discord. Could it be still be done from scratch? What else is different about it?

First I had to find a meaningful thing for the bot to do. I chose Eliza, the classic demonstration of a chatter bot that managed fooling people into having prolonged conversations with them. The version I’m familiar with is M-x doctor which is part of Emacs. So, first of all, I wrote some code to interface with that command in a REPL-style fashion. A companion shell script boots up Emacs in batch mode for interfacing with the doctor from the shell. Much like the original mode, you terminate your input by pressing RET twice. This is an intentional design decision to allow for multi-line input as seen on Discord (but absent from IRC, where you could get away with making it single-line input).

I briefly entertained the thought of writing the rest of the bot from scratch in Emacs Lisp, but abandoned it after learning that I’d need to use websockets with zlib compression to subscribe and respond to incoming messages. While there is an existing library for websocket support, I’d rather not figure out its nitty-gritty details, let alone with the lack of zlib compression. It doesn’t help that Discord’s official API docs are inconclusive and fail answering questions such as how you can set your current status (and more importantly, why it fails getting updated). So, an officially recommended Discord library it is.

The choice on which one it’s going to be depended on whether the programming language it’s implemented with allowed me to communicate with my shell script. I tried out discord.js first, battled a fair bit with Node.js, but gave up eventually. There doesn’t seem to be a way to spawn a child process and read from / write to its stdout / stdin pipes as you see fit. Instead you can only add a callback for the process output and good luck if you want to figure out what piece of output corresponds to the input you wrote earlier. This is why I went for discordrb instead, wrote some glue code for subprocess communication and started figuring out their API to react to incoming messages.

There are a few lessons to be learned from their API:

  • Allow adding as many event handlers as you want for specific events, with convenience options for narrowing down to commonly needed situations (like messages starting with a prefix)
  • Inside these event handlers, provide an object containing all context you’d need, including the information who to respond to
  • Keep the bot alive when an event handler errors out

Now, to test the bot I needed to unleash it on a server. It turns out that unlike on IRC bot accounts are handled specially. You must:

  • Register an application and obtain an ID for authorization purposes
  • Enable a bot account and obtain an authorization token
  • Generate an URL for inviting the bot to a server
  • Share that URL with someone wielding the necessary permissions
  • Hope for the best and wait for them to invite the bot

This meant that I had to create my own test server to check whether my code worked at all. For this reason I haven’t been able to get it running on the server I was invited on. If you want to, you can try it on your own server, the sources are on GitHub as always.

-1:-- Bridging the Ancient and the Modern (Post Vasilij Schneidermann)--L0--C0--March 23, 2018 08:53 AM

Bryan Murdock: It Is Time To Replace Passwords With Keys

It is time to stop using passwords. The Troy Hunt article, Passwords Evolved: Authentication Guidance for the Modern Era just got passed around the office again and wow, what a mess we are in. Instead of passwords we should switch to using public-key cryptography like TLS and ssh use.

"But people can barely manage passwords, there's no way they can manage keys!"

Wrong. People can't barely manage passwords, they can't manage them at all. Read the Troy Hunt article where he says, "Embrace Password Managers." We've given up on humans managing passwords and we are now relying on software to do it. We use software that securely stores our passwords and synchronizes them over the internet so that we have those passwords on all our devices. Often the software generates the passwords for us too. Guess what we could do with keys? The exact same thing.

Now read the Troy Hunt article again and this time think about how it's not just end-users that can't manage passwords, but how developers also cannot manage passwords. There are UI problems, there are concerns with code injection attacks, there are problems with hashing, salting, and storing passwords and with protecting those stored passwords from thieves. Those problems all go away if developers only have to store public keys. No hashing, no salting, no secrets. Think about it. As a user the only thing you'll give to a website to authenticate is your public key. You don't have to trust them with any secrets at all. From the point of view of a developer, you don't have to worry about keeping your customers secrets safe anymore. What a relief! As for UI, if we do it right websites and apps don't need any UI at all for authentication. What is the UI for TLS? The lock symbol that the browser displays. That's it! Websites you visit authenticate themselves to you using strong public-key cryptography behind the scenes, under the covers. It could be the same with user authentication.

Now read the article one more time and think about how not only are users and developers unable to deal with passwords, but security experts can't either. They can't agree on what makes a good secure password, what format it should be in. Special characters? Random strings of characters? Long passphrases of regular words? What's easier to create? Easier to type? Easier to memorize? Should a website show the password as users type it or not? How often should we change passwords? If we are giving up on memorizing and using password managers, does any of that matter? Maybe?

Now think about public-key cryptography. Security experts generally all agree on what makes good public-key cryptography, what format the keys should be in and what length. That was all hashed out years ago. True, there might be disagreement on when stronger keys should be used, whether to use RSA or ECC, and if ECC which curves to use and so on, but regular users relying on key manager software don't have to be involved in those discussions. They don't have to worry anymore about whether they should use a special character or how long of a password to use or if passwords made up of song lyrics are a good idea or not. The discussions on key size or which ECC curve to use raises the debate way beyond trying to figure out what is user friendly but still defeats rainbow tables and script kiddies. It takes the debate up to the level of wondering which nation state might attack you. If we eliminate human involvement in coming up with authentication tokens and remove script kiddies from the attack surface altogether that's a *huge* improvement over passwords.

"OK, but if we use public-key cryptography we also need to use full PKI like TLS."

Do we? PKI provides identity confirmation and key revocation. Do we have identity confirmation for account passwords today? When I create an account with, say, Amazon do they verify that I really am who I say I am? Nope. Not at all. They don't care one bit about who I really am. They just want my money. What about key revocation? Do we have password revocation today? Again, other than manually logging into a website and changing your password, we don't, and we can easily duplicate that manual process with keys in order to get us started. If we don't have full PKI with our public-key authentication we are no worse off than we are today.

The great thing about switching to public-key cryptography is that someday we could add in some sort of Let's Encrypt-like easy-to-use PKI if we want, which would take us light-years beyond where we are with passwords today. We aren't going to get there though if we don't take the first step of ditching passwords.

Getting rid of password authentication and using public-key cryptography instead will make user authentication easier for users, developers, and security experts, and it will make us all more secure.
-1:-- It Is Time To Replace Passwords With Keys (Post Bryan (noreply@blogger.com))--L0--C0--March 23, 2018 04:46 AM

Jonas Bernoulli: Ghub 2.0 and Glab 2.0 released

I am excited to announce the release of Ghub v2.0 and Glab v2.0.
-1:-- Ghub 2.0 and Glab 2.0 released (Post)--L0--C0--March 20, 2018 07:10 PM