Alex's Slip-box

These are my org-mode notes in sort of Zettelkasten style

publish.org

:ID: 33D6368F-C063-40E0-8369-9FA8954C8A46

# Overview

This is a self-documenting org-mode publishing script. It is run by executing all the source code blocks herein. The actual publish.el script just loads this file in a buffer and calls org-babel-execute-buffer followed by the function that starts the publishing process.

# Learning resources for org-mode publishing

# Usage

# Development

# Assets

Assets like CSS and JS files are stored in a git backed directory site_asstes. The files therein are symlinked in the public directory which is gitignored. ln -s site_assets/js public/js ln -s site_assets/css public/css ln -s site_assets/favicon public/favicon

# Serving locally

After building locally (see below) to the public directory, I use the emacs-web-server package to serve it locally. Use httpd-serve-directory and point it at the publishing directory. It will serve the site on http://localhost:8080/. See also Zero config HTTP servers for other options.

# Building

Run build_script.sh. Among other things, this calls out to functions defined in publish.el, which itself uses this file.

This build requires having Ruby and lunr installed (npm install). I use these to pre-build the search index.

Files will build into the public/ dir

In order to not have to mess with build dependencies, use docker:

# With docker

  • Build a tagged container: docker build -t build-site .
  • Run the build script against the container: docker run -it --rm -v "$PWD/public":/app/public build-site

# Deploying

# Dependencies

# Package repositories

 1: (require 'package)
 2: 
 3: (setq package-user-dir (expand-file-name "./.packages"))
 4: 
 5: 
 6: (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
 7: (add-to-list 'package-archives '("elpa" . "https://elpa.gnu.org/packages/") t)
 8: 
 9: (setq package-check-signature nil)
10: 
11: (package-initialize)

# use-package

See use-package repo for more on this. It’s a clean way to handle dependencies in emacs.

12: (unless (package-installed-p 'use-package)
13:   (package-install 'use-package))
14: (require 'use-package)

# esxml

esxml provides elisp functions to generate HTML markup. Basically this means less reliance on ugly concat and format function calls.

Breaking Change Using esxml below is currently broken due to a change in esxml that escapes HTML when inserting strings. See this issue. Using a function to render the header, renders a string that gets HTML escaped when inserted into the main template below. Hence, the HTML is broken in this case and others that follow this pattern.

To temporarily get around that, I’ve “locked” the esxml package by copying a working version of it to .packages as part of the build process. Then it gets manually loaded here if it exists. Kinda gross, but it works for now.

15: (let ((path (car (file-expand-wildcards "./.packages/esxml*/esxml.el"))))
16:   (if (file-exists-p path)
17:       (load-file path)))
18: (use-package esxml :ensure t)

# htmlize

  • I don’t really know much about emacs-htmlize and all of its capabilities, but in the context of this script, it provides CSS styling for code syntax highlighting.
  • I believe the default is to use inline CSS, but it can generate a style sheet based on your emacs theme by calling org-html-htmlize-generate-css. I did that then linked the stylesheet in the HTML document <head>.
  • Tell it to use a stylesheet over line styles by setting the org-html-htmlize-output-type variable. See below.
  • Check out Org css for more on this.
19: (use-package htmlize :ensure t)

# ts

ts.el for sanity when formatting and parsing dates.

20: (use-package ts :ensure t)

# s

s.el for sanity when working with strings.

21: (use-package s :ensure t)

# ox-publish

The publishing system for org-mode

22: (require 'ox-publish)

# Variables

# Site variables

These get referenced when generating the HTML.

23: (setq my/site-title   "Alex's Slip-box"
24:       my/site-tagline "These are my org-mode notes in sort of Zettelkasten style"
25:       my/sitemap-title "")

# Org publish and export variables

I’m not going to bother explaining all these since they’re thoroughly explained with describe-variable

26: (setq org-publish-use-timestamps-flag t
27:       org-publish-timestamp-directory "./.org-cache/"
28:       org-export-with-section-numbers nil
29:       org-export-use-babel nil
30:       org-export-with-smart-quotes t
31:       org-export-with-sub-superscripts nil
32:       org-export-with-tags 'not-in-toc
33:       org-export-date-timestamp-format "Y-%m-%d %H:%M %p"
34:       org-id-locations-file-relative t
35:       org-id-locations-file "./.org-id-locations"
36:       org-id-track-globally t)

# HTML exporter variables

  • Tell htmlize to use a CSS stylesheet rather than inline styles.
  • Use describe-variable to learn about the rest of them.
37: (setq org-html-metadata-timestamp-format "%Y-%m-%d"
38:       org-html-checkbox-type 'site-html
39:       org-html-html5-fancy nil
40:       org-html-htmlize-output-type 'css
41:       org-html-self-link-headlines t
42:       org-html-validation-link nil
43:       org-html-inline-images t
44:       org-html-doctype "html5")

# Other variables

This is backed by a git repository, so we don’t need backups

45: (setq make-backup-files nil)

# Export document

# Site header

  • This function is called when generating the HTML template below.
    • info arg is a plist from which we can get configuration details about the org document. I’m not using it here, but it comes in handy in other functions to get things like the document title, date, etc.
  • Here I am using esxml to declare the markup in elisp.
    • It’s quoted (with `) but we can use , to selectively evaluate expressions therein. Noice.
    • @ function is for declaring node attributes like class, id or whatever.
46: (defun my/site-header (info)
47:   (sxml-to-xml
48:    `(div (@ (class "header uk-section uk-section-primary"))
49:          (div (@ (class "heading uk-container"))
50:               (div (@ (class "site-title-container uk-flex uk-flex-middle"))
51:                    (h1 (@ (class "site-title uk-h1 uk-heading-medium")) ,my/site-title)
52:                    (form (@ (class "uk-search uk-search-default search-form")
53:                             (id "search-form"))
54:                          (span (@ (uk-search-icon "")) "")
55:                          (input (@ (class "uk-search-input")
56:                                    (type "search")
57:                                    (placeholder "Search")))))
58:               (div (@ (class "site-tagline uk-text-lead")) ,my/site-tagline))
59:          (div (@ (class "uk-container"))
60:               (nav (@ (class "uk-navbar-container uk-navbar-transparent")
61:                       (uk-navbar))
62:                    (div (@ (class "uk-navbar-left"))
63:                         (ul (@ (class "uk-navbar-nav"))
64:                             (li (a (@ (class "nav-link") (href "/")) "Notes"))
65:                             (li (a (@ (class "nav-link") (href "https://github.com/apmiller108")) "Github"))
66:                             (li (a (@ (class "nav-link") (href "https://alex-miller.co")) "alex-miller.co")))))))))

# Site footer

  • This function is called when generating the HTML template below.
  • creator is Emacs {{version}} (Org mode {{version}})
67: (defun my/site-footer (info)
68:   (sxml-to-xml
69:   `(footer (@ (class "footer uk-section uk-section-secondary"))
70:             (div (@ (class "uk-container footer-container"))
71:                  (div (@ (class "footer-links"))
72:                       (a (@ (href "https://github.com/apmiller108")
73:                             (class "footer-link")
74:                             (uk-icon "icon: github-alt"))
75:                             "github")
76:                       (a (@ (href "https://notes.alex-miller.co/")
77:                             (class "footer-link")
78:                             (uk-icon "icon: album"))
79:                             "notes")
80:                       (a (@ (href "https://alex-miller.co/")
81:                             (class "footer-link")
82:                             (uk-icon "icon: home"))
83:                             "alex-miller.co"))
84:                  (div (@ (class "copyright"))
85:                       (p "Made with " ,(plist-get info :creator))
86:                       (p ,(format "Copyright © %d Alex P. Miller. All rights reserved." (string-to-number (format-time-string "%Y")))))))))

# The HTML Template

  • This is the whole page layout. It makes use of the header and footer functions above and injects the org-mode document exported HTML (the contents arg).
  • I think all of this is pretty self explanatory, but one thing I should call out is the use of :filetags to generate the tag links. I’m not entirely sure I had to do this, but I declared as a custom export option in the derived backend. See below.
  • Same with the :updated property.
 87: (defun my/org-html-template (contents info)
 88:   (concat
 89:   "<!DOCTYPE html>"
 90:   (sxml-to-xml
 91:     `(html (@ (lang "en"))
 92:           (head
 93:             (meta (@ (charset "utf-8")))
 94:             (meta (@ (author "Alex P. Miller")))
 95:             (meta (@ (name "viewport")
 96:                     (content "width=device-width, initial-scale=1, shrink-to-fit=no")))
 97:             (link (@ (rel "apple-touch-icon")
 98:                     (sizes "180x180")
 99:                     (href "/favicon/apple-touch-icon.png?v=1")))
100:             (link (@ (rel "icon")
101:                     (type "image/png")
102:                     (sizes "32x32")
103:                     (href "/favicon/favicon-32x32.png?v=1")))
104:             (link (@ (rel "icon")
105:                     (type "image/png")
106:                     (sizes "16x16")
107:                     (href "/favicon/favicon-16x16.png?v=1")))
108:             (link (@ (rel "manifest")
109:                     (href "/favicon/manifest.json?v=1")))
110:             (link (@ (rel "mask-icon")
111:                     (href "/favicon/safari-pinned-tab.svg?v=1")))
112:             (link (@ (rel "stylesheet")
113:                     (href "/css/uikit.min.css")))
114:             (link (@ (rel "stylesheet")
115:                     (href "/css/code.css")))
116:             (link (@ (rel "stylesheet")
117:                     (href "/css/site.css")))
118:             (script (@ (src "/js/uikit.min.js")) nil)
119:             (script (@ (src "/js/uikit-icons.min.js")) nil)
120:             (script (@ (src "/js/lunr.min.js")) nil)
121:             (script (@ (src "/js/site.js")) nil)
122:             (script (@ (src "https://www.googletagmanager.com/gtag/js?id=G-YM3EHHB2YQ")) nil)
123:             (script
124:             "window.dataLayer = window.dataLayer || [];
125:               function gtag(){dataLayer.push(arguments);}
126:               gtag('js', new Date());
127:               gtag('config', 'G-YM3EHHB2YQ');"
128:             )
129:             (title ,(concat (org-export-data (plist-get info :title) info) " - Alex's Notes")))
130:           (body
131:             ,(my/site-header info)
132:             (div (@ (class "main uk-section uk-section-muted"))
133:                   (div (@ (class "note uk-container"))
134:                       (div (@ (class "note-content"))
135:                             (h1 (@ (class "note-title uk-h1"))
136:                                 ,(org-export-data (plist-get info :title) info))
137:                             (div (@ (class "note-meta"))
138:                                 ,(when (plist-get info :date)
139:                                     `(p (@ (class "note-created uk-article-meta"))
140:                                         ,(format "Created on %s" (ts-format "%B %e, %Y" (ts-parse (org-export-data (plist-get info :date) info))))))
141:                                 ,(when (plist-get info :updated)
142:                                     `(p (@ (class "note-updated uk-article-meta"))
143:                                         ,(format "Updated on %s" (ts-format "%B %e, %Y" (ts-parse (plist-get info :updated)))))))
144:                             ,(let ((tags (org-export-data (plist-get info :filetags) info)))
145:                                (when (and tags (> (length tags) 0))
146:                                  `(p (@ (class "blog-post-tags"))
147:                                      "Tags: "
148:                                      ,(mapconcat (lambda (tag) (format "<a href=\"/?tag=%s\">%s</a>" tag tag))
149:                                                  (plist-get info :filetags)
150:                                                  ", "))))
151:                             ,contents)
152:                       ,(when (not (string-equal my/sitemap-title (org-export-data (plist-get info :title) info)))
153:                           '(script (@ (src "https://utteranc.es/client.js")
154:                                       (repo "apmiller108/slip-box")
155:                                       (issue-term "title")
156:                                       (label "comments")
157:                                       (theme "boxy-light")
158:                                       (crossorigin "anonymous")
159:                                       (async))
160:                                   nil))))
161:                   ,(my/site-footer info)
162:                   (div (@ (id "search-results")
163:                           (class "search-results")
164:                           (uk-modal ""))
165:                        (div (@ (class "uk-modal-dialog uk-modal-body"))
166:                             (h2 (@ (class "uk-modal-title")
167:                                    (id "search-results-title"))
168:                                 "Search Results")
169:                             (div (@ (id "search-results-body")
170:                                     (class "search-results-body"))
171:                                  "")
172:                             (a (@ (class "uk-modal-close-default")
173:                                        (uk-close ""))
174:                                     ""))))))))

# Element customization

# Links and Images

  • The link paths need to match the actual file paths of the exported files. So for file links, the exported link paths are downcased and without filename extensions. So, this function ensures the link paths match that format. So [[file:my_post.org][My Post]] becomes <a href="my_post">My Post</a> (no .html on the path).
  • Org-roam uses the ID property for linking notes (ie, no file path). To get around this I do the following:
    1. In my my publish.el file, I generate the .org-id-locations file. This file is committed since it is also used on CI where I couldn’t even generate this file as part of the build process.
    2. Again in publish.el , set the my/org-id-locations variable to a hashtable generated from the .org-id-locations file.
    3. For fuzzy type links, find the path from the hashtable. Oh, somehow the fuzzy type links are the ID links.
    4. Seriously, what a pain in the arse.
  • I have some inline images in my org files. These are file links without a label that point to files with image extensions. Mostly these are plantuml renderings. They get converted to HTML img tags.
  • For everything else, just render a good old fashion anchor tag.
175: (defun my/format-path-for-anchor-tag (path)
176:   (concat "/"
177:           (downcase
178:            (file-name-sans-extension
179:             path))))
180: (defun my/org-html-link (link contents info)
181:   "Removes file extension and changes the path into lowercase org file:// links.
182:   Handles creating inline images with `<img>' tags for png, jpg, and svg files
183:   when the link doesn't have a label, otherwise just creates a link."
184:   ;; TODO: refactor this mess
185:   (if (string= 'fuzzy (org-element-property :type link))
186:       (let ((path (gethash (s-replace "id:" "" (org-element-property :path link)) my/org-id-locations)))
187:         (if path
188:             (org-element-put-property link :path
189:                                       (my/format-path-for-anchor-tag
190:                                        (car (last (s-split "/" path))))))))
191:   (when (and (string= 'file (org-element-property :type link))
192:             (string= "org" (file-name-extension (org-element-property :path link))))
193:     (org-element-put-property link :path
194:                               (my/format-path-for-anchor-tag
195:                                         (org-element-property :path link))))
196: 
197:   (if (and (string= 'file (org-element-property :type link))
198:           (file-name-extension (org-element-property :path link))
199:           (string-match "png\\|jpg\\|svg\\|webp"
200:                         (file-name-extension
201:                           (org-element-property :path link)))
202:           (equal contents nil))
203:       (format "<img src=/%s >" (org-element-property :path link))
204:     (if (and (equal contents nil)
205:             (or (not (file-name-extension (org-element-property :path link)))
206:                 (and (file-name-extension (org-element-property :path link))
207:                       (not (string-match "png\\|jpg\\|svg\\|webp"
208:                                         (file-name-extension
209:                                           (org-element-property :path link)))))))
210:         (format "<a href=\"%s\">%s</a>"
211:                 (org-element-property :raw-link link)
212:                 (org-element-property :raw-link link))
213:       (format "<a href=\"%s\">%s</a>"
214:               (org-element-property :path link)
215:               contents))))

# Headings

This part is largely unchanged from David Wilson’s publish.el on which this is based.

  • Maybe something else already requires subx-r.el, but we make sure we can use thread-last.
  • This helper function is used when rendering headlines. It kebab cases the cases the headline text for use as the HTML element’s ID.
    • Sometimes heading words are fenced with ~, so the code tag is removed.
216: (require 'subr-x)
217: 
218: (defun my/make-heading-anchor-name (headline-text)
219:   (thread-last headline-text
220:     (downcase)
221:     (replace-regexp-in-string " " "-")
222:     (replace-regexp-in-string "</?code>" "")
223:     (replace-regexp-in-string "[^[:alnum:]_]" "")))
  • Basically, this translates the org-mode headlines to HTML h tags of the corresponding level with anchor tag handles, IDs that can be easily linked to, while respecting export options.
224: (defun my/org-html-headline (headline contents info)
225:   (let* ((text (org-export-data (org-element-property :title headline) info))
226:         (level (org-export-get-relative-level headline info))
227:         (level (min 7 (when level (1+ level))))
228:         (anchor-name (my/make-heading-anchor-name text))
229:         (attributes (org-element-property :ATTR_HTML headline))
230:         (container (org-element-property :HTML_CONTAINER headline))
231:         (container-class (and container (org-element-property :HTML_CONTAINER_CLASS headline))))
232:     (when attributes
233:       (setq attributes
234:             (format " %s" (org-html--make-attribute-string
235:                            (org-export-read-attribute 'attr_html
236:                                                       `(nil
237:                                                         (attr_html ,(split-string attributes))))))))
238:     (concat
239:      (when (and container (not (string= "" container)))
240:        (format "<%s%s>" container (if container-class (format " class=\"%s\"" container-class) "")))
241:      (if (not (org-export-low-level-p headline info))
242:          (format "<h%d%s><a id=\"%s\" class=\"anchor\" href=\"#%s\"><i># </i></a>%s</h%d>%s"
243:                 level
244:                 (or attributes "")
245:                 anchor-name
246:                 anchor-name
247:                 text
248:                 level
249:                 (or contents ""))
250:        (concat
251:         (when (org-export-first-sibling-p headline info) "<ul>")
252:         (format "<li>%s%s</li>" text (or contents ""))
253:         (when (org-export-last-sibling-p headline info) "</ul>")))
254:      (when (and container (not (string= "" container)))
255:        (format "</%s>" (cl-subseq container 0 (cl-search " " container)))))))

# The Sitemap (the home page)

# Sitemap Entry

Formats sitemap entry as {date} {title} ({filetags}). Returns a list containing the sitemap entry string and the filetags. A unique list of the filetags is created on the sitemap page from this list, that’s why they’re returned from this function.

256: (defun my/sitemap-format-entry (entry style project)
257:   (let* ((filetags (org-publish-find-property entry :filetags project 'site-html))
258:          (created-at (format-time-string "%Y-%m-%d"
259:                                          (date-to-time
260:                                           (format "%s" (car (org-publish-find-property entry :date project))))))
261:          (entry
262:           (sxml-to-xml
263:            `(li (@ (data-date ,created-at)
264:                    (class ,(mapconcat (lambda (tag) tag) filetags " ")))
265:                 (span (@ (class "sitemap-entry-date")) ,created-at)
266:                 (a (@ (href ,(file-name-sans-extension entry)))
267:                    ,(org-publish-find-title entry project))
268: 
269:                 ,(if filetags
270:                      `(span (@ (class "sitemap-entry-tags"))
271:                             ,(concat "("
272:                                      (mapconcat (lambda (tag) tag) filetags ", ")
273:                                      ")")))))))
274:         (list entry filetags)))

# Sitemap page

From the function above, the filetags are placed into a flattened list, duplicate values removed and sorted alphabetical ascending. These are turned into tags on the page used for filtering the entries by topic. All of the JS used for filtering is provided by the UIkit CSS framework.

275: (defun my/sitemap (title list)
276:   (let* ((unique-tags
277:           (sort
278:           (delete-dups
279:             (flatten-tree
280:               (mapcar (lambda (item) (cdr (car item)))
281:                       (cdr list))))
282:           (lambda (a b) (string< a b)))))
283:     (concat
284:     "#+TITLE: " title "\n\n"
285:     "#+BEGIN_EXPORT html\n\n"
286:     (sxml-to-xml
287:      `(div (@ (id "tag-filter-component")
288:               (uk-filter "target: .js-filter"))
289:            (div (@ (class "tags uk-subnav uk-subnav-pill"))
290:                 (span (@ (uk-filter-control "group: tag"))
291:                       (a (@ (href "#")) "ALL"))
292:                 ,(mapconcat (lambda (item)
293:                               (format "<span id=\"%s\" uk-filter-control=\"filter: .%s; group: tag\"><a href=\"#\">%s</a></span>"
294:                                       (concat "filter-" item)
295:                                       item
296:                                       item))
297:                             unique-tags
298:                             "\n"))
299:            (ul (@ (class "uk-subnav uk-subnav-pill"))
300:                (li (@ (uk-filter-control "sort: data-date; group: date"))
301:                    (a (@ (href "#")) "Ascending"))
302:                (li (@ (uk-filter-control "sort: data-date; order: desc; group: date")
303:                       (class "uk-active"))
304:                    (a (@ (href "#")) "Descending")))
305:            (ul (@ (class "sitemap-entries uk-list uk-list-emphasis js-filter"))
306:                ,(mapconcat (lambda (item) (car (car item)))
307:                           (cdr list)
308:                           "\n"))))
309:     "\n#+END_EXPORT\n")))

# Derived backend

You can derive a custom backend from an existing one and can override certain functions. In this example, my-site-html derives from html and overrides template, link, and headline functions.

310: (org-export-define-derived-backend
311:     'site-html
312:     'html
313:   :translate-alist
314:   '((template . my/org-html-template)
315:     (link . my/org-html-link)
316:     (headline . my/org-html-headline))
317:   :options-alist
318:   '((:page-type "PAGE-TYPE" nil nil t)
319:     (:html-use-infojs nil nil nil)
320:     (:updated "UPDATED" nil nil t)
321:     (:filetags "FILETAGS" nil nil split)))

# Publishing

# Output paths

This is a helper function that converts an org-mode file name to a directory of the same name, downcased and without the filename extension. So if the filename is my-post.org, a sub-directory would be created in the publishing directory called my-post/. The sitemap is indented to be at the root of the publishing directory (ie, the homepage). This function is called in the next code block.

322: (defun get-article-output-path (org-file pub-dir)
323:   (let ((article-dir (concat pub-dir
324:                             (downcase
325:                               (file-name-as-directory
326:                               (file-name-sans-extension
327:                                 (file-name-nondirectory org-file)))))))
328:     (if (string-match "\\/sitemap.org$" org-file)
329:         pub-dir
330:         (progn
331:           (unless (file-directory-p article-dir)
332:             (make-directory article-dir t))
333:           article-dir))
334:     ))

# The publishing function (and conditional TOCs)

This function does a few things:

  • It adds the export option to generate a table of contents only if there are more than 3 headlines. Otherwise, I don’t see a point to rendering a TOC.
  • Next it calls the helper function above to create the output directory and appends index.html to the result. This ends up being the article-path for a post. For example, if the filename is my-post.org, the article path would be /my-post/index.html.
  • Finally, it calls org-publish-org-to which publishes a file using the selected backend.
335: (defun my/org-html-publish-to-html (plist filename pub-dir)
336:   (with-current-buffer (find-file filename)
337:     (when (> (length (org-map-entries t)) 3)
338:       (insert "#+OPTIONS: toc:t\n")))
339:   (let ((article-path (get-article-output-path filename pub-dir)))
340:     (cl-letf (((symbol-function 'org-export-output-file-name)
341:               (lambda (extension &optional subtreep pub-dir)
342:                 (concat article-path "index" extension))))
343:       (org-publish-org-to 'site-html
344:                           filename
345:                           (concat "." (or (plist-get plist :html-extension) "html"))
346:                           plist
347:                           article-path))))
348: 

# The project alist

This is the configuration for the publishable projects. Each project can be published independently with org-publish and the project name (eg (org-publish "site")), or all of them with org-publish-all.

349: (setq org-publish-project-alist
350:       (list
351:        (list "notes.alex-miller.co"
352:              :base-extension "org"
353:              :base-directory "./"
354:              :publishing-function '(my/org-html-publish-to-html)
355:              :publishing-directory "./public"
356:              :auto-sitemap t
357:              :sitemap-function 'my/sitemap
358:              :sitemap-title my/sitemap-title
359:              :sitemap-format-entry 'my/sitemap-format-entry
360:              :sitemap-sort-files 'alphabetically
361:              :with-title nil
362:              :with-toc nil)
363:        (list "images"
364:              :base-extension "png\\|jpg\\|svg\\|webp"
365:              :base-directory "./images"
366:              :publishing-directory "./public/images"
367:              :publishing-function 'org-publish-attachment)
368:        (list "site" :components '("notes.alex-miller.co" "images"))))

# notes.alex-miller.co

This publishes the org-mode files. I keep them in the root directory. I have a few other folders for other note types that I don’t publish. The HTML output is placed in the ./public directory which is gitignored. The sitemap functions are documented above. TOCs are only generated for notes that have more than 3 headlines.

# images

I sometimes link and display images in my org-notes, like plantuml renderings. I put these in the ./images directory. This basically just copies them over to the /public/images directory of the site. This ensure that links and/or inline images work. (See this emacs.stackexchange answer for where I got the idea).

# site

It contains everything needed to build the site.

Search Results