Alex's Slip-box

These are my org-mode notes in sort of Zettelkasten style

publish.org

# Overview

This is a self-documenting org-mode publishing script. It is run by executing all the source code blocks herein. The actual publish.el script just loads this file in a buffer and calls org-babel-execute-buffer followed by the function that starts the publishing process.

# Learning resources for org-mode publishing

# Usage

emacs -Q --batch -l ./publish.el --funcall my/publish

# Development

# Assets

Assets like CSS and JS files are stored in a git backed directory site_asstes. The files therein are symlinked in the publish directory which is gitignored.

# Serving locally

After building locally to the public directory, I use the emacs-web-server package to serve it locally. Use httpd-serve-directory and point it at the publishing directory. It will serve the site on http://localhost:8080/.

# Dependencies

# Package repositories

  • Adds melpa and elpa archives.
  • Sets the package archives directory so that packages aren’t installed in ~/.emacs.d/elpa.
 1: (require 'package)
 2: 
 3: (setq package-user-dir (expand-file-name "./.packages"))
 4: 
 5: (setq package-archives '(("melpa" . "https://melpa.org/packages/")
 6:                         ("elpa" . "https://elpa.gnu.org/packages/")))
 7: 
 8: (package-initialize)
 9: (unless package-archive-contents
10:   (package-refresh-contents))

# use-package

See use-package repo for more on this. It’s a clean way to handle dependencies in emacs.

11: (unless (package-installed-p 'use-package)
12:   (package-install 'use-package))
13: (require 'use-package)

# esxml

esxml provides elisp functions to generate HTML markup. Basically this means less reliance on ugly concat and format function calls.

14: (use-package esxml :ensure t)

# htmlize

  • I don’t really know much about emacs-htmlize and all of its capabilities, but in the context of this script, it provides CSS styling for code syntax highlighting.
  • I believe the default is to use inline CSS, but it can generate a style sheet based on your emacs theme by calling org-html-htmlize-generate-css. I did that then linked the stylesheet in the HTML document <head>.
  • Tell it to use a stylesheet over line styles by setting the org-html-htmlize-output-type variable. See below.
  • Check out Org css for more on this.
15: (use-package htmlize :ensure t)

# ts

ts.el for sanity when formatting and parsing dates.

16: (use-package ts :ensure t)

# ox-publish

The publishing system for org-mode

17: (require 'ox-publish)

# Variables

# Site variables

These get referenced when generating the HTML.

18: (setq my/site-title   "Alex's Slip-box"
19:       my/site-tagline "These are my org-mode notes in sort of Zettelkasten style"
20:       my/sitemap-title "")

# Org publish and export variables

I’m not going to bother explaining all these since they’re thoroughly explained with describe-variable

21: (setq org-publish-use-timestamps-flag t
22:       org-publish-timestamp-directory "./.org-cache/"
23:       org-export-with-section-numbers nil
24:       org-export-use-babel nil
25:       org-export-with-smart-quotes t
26:       org-export-with-sub-superscripts nil
27:       org-export-with-tags 'not-in-toc
28:       org-export-date-timestamp-format "Y-%m-%d %H:%M %p")

# HTML exporter variables

  • Tell htmlize to use a CSS stylesheet rather than inline styles.
  • Use describe-variable to learn about the rest of them.
29: (setq org-html-metadata-timestamp-format "%Y-%m-%d"
30:       org-html-checkbox-type 'site-html
31:       org-html-html5-fancy nil
32:       org-html-htmlize-output-type 'css
33:       org-html-self-link-headlines t
34:       org-html-validation-link nil
35:       org-html-inline-images t
36:       org-html-doctype "html5")

# Other variables

This is backed by a git repository, so we don’t need backups

37: (setq make-backup-files nil)

# Export document

# Site header

  • This function is called when generating the HTML template below.
    • info arg is a plist from which we can get configuration details about the org document. I’m not using it here, but it comes in handy in other functions to get things like the document title, date, etc.
  • Here I am using esxml to declare the markup in elisp.
    • It’s quoted (with `) but we can use , to selectively evaluate expressions therein. Noice.
    • @ function is for declaring node attributes like class, id or whatever.
38: (defun my/site-header (info)
39:   (sxml-to-xml
40:    `(div (@ (class "header uk-section uk-section-primary"))
41:          (div (@ (class "heading uk-container"))
42:               (div (@ (class "site-title-container uk-flex uk-flex-middle"))
43:                    (h1 (@ (class "site-title uk-h1 uk-heading-medium")) ,my/site-title))
44:               (div (@ (class "site-tagline uk-text-lead")) ,my/site-tagline))
45:          (div (@ (class "uk-container"))
46:               (nav (@ (class "uk-navbar-container uk-navbar-transparent")
47:                       (uk-navbar))
48:                    (div (@ (class "uk-navbar-left"))
49:                         (ul (@ (class "uk-navbar-nav"))
50:                             (li (a (@ (class "nav-link") (href "/")) "Notes"))
51:                             (li (a (@ (class "nav-link") (href "https://github.com/apmiller108")) "Github"))
52:                             (li (a (@ (class "nav-link") (href "https://alex-miller.co")) "alex-miller.co")))))))))

# Site footer

  • This function is called when generating the HTML template below.
  • creator is Emacs {{version}} (Org mode {{version}})
53: (defun my/site-footer (info)
54:   (sxml-to-xml
55:   `(footer (@ (class "footer uk-section uk-section-secondary"))
56:             (div (@ (class "uk-container footer-container"))
57:                  (div (@ (class "footer-links"))
58:                       (a (@ (href "https://notes.alex-miller.co/")
59:                             (class "footer-link")
60:                             (uk-icon "icon: album"))
61:                             "notes")
62:                       (a (@ (href "https://github.com/apmiller108")
63:                             (class "footer-link")
64:                             (uk-icon "icon: github-alt"))
65:                             "github")
66:                       (a (@ (href "https://twitter.com/apmiller108")
67:                             (class "footer-link")
68:                             (uk-icon "icon: twitter"))
69:                          "@apmiller108")
70:                       (a (@ (href "https://www.reddit.com/user/apmillz")
71:                             (class "footer-link")
72:                             (uk-icon "icon: reddit"))
73:                          "u/apmillz"))
74:                  (div (@ (class "made-with"))
75:                       (p "Made with " ,(plist-get info :creator)))))))

# The HTML Template

  • This is the whole page layout. It makes use of the header and footer functions above and injects the org-mode document exported HTML (the contents arg).
  • I think all of this is pretty self explanatory, but one thing I should call out is the use of :roam_tags to generate the tag links. :roam_tags (from org-roam package) are not automatically available from the info plist. This needs to first be declared as a custom export option in the derived backend. See below. I took me a while to figure that out. I even asked on emacs.stackexchange, but eventually figured it out and answered my own question.
  • Same with the :updated property.
 76: (defun my/org-html-template (contents info)
 77:   (concat
 78:   "<!DOCTYPE html>"
 79:   (sxml-to-xml
 80:     `(html (@ (lang "en"))
 81:           (head
 82:             (meta (@ (charset "utf-8")))
 83:             (meta (@ (author "Alex P. Miller")))
 84:             (meta (@ (name "viewport")
 85:                     (content "width=device-width, initial-scale=1, shrink-to-fit=no")))
 86:             (link (@ (rel "apple-touch-icon")
 87:                     (sizes "180x180")
 88:                     (href "/favicon/apple-touch-icon.png?v=1")))
 89:             (link (@ (rel "icon")
 90:                     (type "image/png")
 91:                     (sizes "32x32")
 92:                     (href "/favicon/favicon-32x32.png?v=1")))
 93:             (link (@ (rel "icon")
 94:                     (type "image/png")
 95:                     (sizes "16x16")
 96:                     (href "/favicon/favicon-16x16.png?v=1")))
 97:             (link (@ (rel "manifest")
 98:                     (href "/favicon/manifest.json?v=1")))
 99:             (link (@ (rel "mask-icon")
100:                     (href "/favicon/safari-pinned-tab.svg?v=1")))
101:             (link (@ (rel "stylesheet")
102:                     (href "/css/uikit.min.css")))
103:             (link (@ (rel "stylesheet")
104:                     (href "/css/code.css")))
105:             (link (@ (rel "stylesheet")
106:                     (href "/css/site.css")))
107:             (script (@ (src "/js/uikit.min.js")) nil)
108:             (script (@ (src "/js/uikit-icons.min.js")) nil)
109:             (script (@ (src "/js/site.js")) nil)
110:             (script (@ (src "https://www.googletagmanager.com/gtag/js?id=G-YM3EHHB2YQ")) nil)
111:             (script
112:             "window.dataLayer = window.dataLayer || [];
113:               function gtag(){dataLayer.push(arguments);}
114:               gtag('js', new Date());
115: 
116:               gtag('config', 'G-YM3EHHB2YQ');"
117:             )
118:             (title ,(concat (org-export-data (plist-get info :title) info) " - notes.alex-miller.com")))
119:           (body
120:             ,(my/site-header info)
121:             (div (@ (class "main uk-section uk-section-muted"))
122:                   (div (@ (class "note uk-container"))
123:                       (div (@ (class "note-content"))
124:                             (h1 (@ (class "note-title uk-h1"))
125:                                 ,(org-export-data (plist-get info :title) info))
126:                             (div (@ (class "note-meta"))
127:                                 ,(when (plist-get info :date)
128:                                     `(p (@ (class "note-created uk-article-meta"))
129:                                         ,(format "Created on %s" (ts-format "%B %e, %Y" (ts-parse (org-export-data (plist-get info :date) info))))))
130:                                 ,(when (plist-get info :updated)
131:                                     `(p (@ (class "note-updated uk-article-meta"))
132:                                         ,(format "Updated on %s" (ts-format "%B %e, %Y" (ts-parse (plist-get info :updated)))))))
133:                             ,(let ((tags (org-export-data (plist-get info :roam_tags) info)))
134:                                (when (and tags (> (length tags) 0))
135:                                  `(p (@ (class "blog-post-tags"))
136:                                      "Tags: "
137:                                      ,(mapconcat (lambda (tag) (format "<a href=\"/?tag=%s\">%s</a>" tag tag))
138:                                                  (plist-get info :roam_tags)
139:                                                  ", "))))
140:                             ,contents)
141:                       ,(when (not (string-equal my/sitemap-title (org-export-data (plist-get info :title) info)))
142:                           '(script (@ (src "https://utteranc.es/client.js")
143:                                       (repo "apmiller108/slip-box")
144:                                       (issue-term "title")
145:                                       (label "comments")
146:                                       (theme "boxy-light")
147:                                       (crossorigin "anonymous")
148:                                       (async))
149:                                   nil))))
150:                   ,(my/site-footer info))))))

# Element customization

# Links and Images

  • The link paths need to match the actual file paths of the exported files. The exported files are downcased and without filename extensions. So, this function ensures the link paths match that format. So [[file:my_post.org][My Post]] becomes <a href="my_post">My Post</a> (no .html on the path).
  • I have some inline images in my org files. These are file links without a label that point to files with image extensions. Mostly these are plantuml renderings. They get converted to HTML img tags.
  • For everything else, just render a good old fashion anchor tag.
151: (defun my/org-html-link (link contents info)
152:   "Removes file extension and changes the path into lowercase org file:// links.
153:   Handles creating inline images with `<img>' tags for png, jpg, and svg files
154:   when the link doesn't have a label, otherwise just creates a link."
155:   ;; TODO: refactor this mess
156:   (when (and (string= 'file (org-element-property :type link))
157:             (string= "org" (file-name-extension (org-element-property :path link))))
158:     (org-element-put-property link :path
159:                               (concat "/"
160:                                       (downcase
161:                                       (file-name-sans-extension
162:                                         (org-element-property :path link))))))
163: 
164:   (if (and (string= 'file (org-element-property :type link))
165:           (file-name-extension (org-element-property :path link))
166:           (string-match "png\\|jpg\\|svg"
167:                         (file-name-extension
168:                           (org-element-property :path link)))
169:           (equal contents nil))
170:       (format "<img src=/%s >" (org-element-property :path link))
171:     (if (and (equal contents nil)
172:             (or (not (file-name-extension (org-element-property :path link)))
173:                 (and (file-name-extension (org-element-property :path link))
174:                       (not (string-match "png\\|jpg\\|svg"
175:                                         (file-name-extension
176:                                           (org-element-property :path link)))))))
177:         (format "<a href=\"%s\">%s</a>"
178:                 (org-element-property :raw-link link)
179:                 (org-element-property :raw-link link))
180:       (format "<a href=\"%s\">%s</a>"
181:               (org-element-property :path link)
182:               contents))))

# Headings

This part is largely unchanged from David Wilson’s publish.el on which this is based.

  • Maybe something else already requires subx-r.el, but we make sure we can use thread-last.
  • This helper function is used when rendering headlines. It kebab cases the cases the headline text for use as the HTML element’s ID.
    • Sometimes heading words are fenced with ~, so the code tag is removed.
183: (require 'subr-x)
184: 
185: (defun my/make-heading-anchor-name (headline-text)
186:   (thread-last headline-text
187:     (downcase)
188:     (replace-regexp-in-string " " "-")
189:     (replace-regexp-in-string "</?code>" "")
190:     (replace-regexp-in-string "[^[:alnum:]_-]" "")))
  • Basically, this translates the org-mode headlines to HTML h tags of the corresponding level with anchor tag handles, IDs that can be easily linked to, while respecting export options.
191: (defun my/org-html-headline (headline contents info)
192:   (let* ((text (org-export-data (org-element-property :title headline) info))
193:         (level (org-export-get-relative-level headline info))
194:         (level (min 7 (when level (1+ level))))
195:         (anchor-name (my/make-heading-anchor-name text))
196:         (attributes (org-element-property :ATTR_HTML headline))
197:         (container (org-element-property :HTML_CONTAINER headline))
198:         (container-class (and container (org-element-property :HTML_CONTAINER_CLASS headline))))
199:     (when attributes
200:       (setq attributes
201:             (format " %s" (org-html--make-attribute-string
202:                            (org-export-read-attribute 'attr_html
203:                                                       `(nil
204:                                                         (attr_html ,(split-string attributes))))))))
205:     (concat
206:      (when (and container (not (string= "" container)))
207:        (format "<%s%s>" container (if container-class (format " class=\"%s\"" container-class) "")))
208:      (if (not (org-export-low-level-p headline info))
209:          (format "<h%d%s><a id=\"%s\" class=\"anchor\" href=\"#%s\"><i># </i></a>%s</h%d>%s"
210:                 level
211:                 (or attributes "")
212:                 anchor-name
213:                 anchor-name
214:                 text
215:                 level
216:                 (or contents ""))
217:        (concat
218:         (when (org-export-first-sibling-p headline info) "<ul>")
219:         (format "<li>%s%s</li>" text (or contents ""))
220:         (when (org-export-last-sibling-p headline info) "</ul>")))
221:      (when (and container (not (string= "" container)))
222:        (format "</%s>" (cl-subseq container 0 (cl-search " " container)))))))

# The Sitemap (the home page)

# Sitemap Entry

Formats sitemap entry as {date} {title} ({roam_tags}). Returns a list containing the sitemap entry string and the roam_tags. A unique list of the roam_tags is created on the sitemap page from this list, that’s why they’re returned from this function.

223: (defun my/sitemap-format-entry (entry style project)
224:   (let* ((roam-tags (org-publish-find-property entry :roam_tags project 'site-html))
225:          (created-at (format-time-string "%Y-%m-%d"
226:                                          (date-to-time
227:                                           (format "%s" (car (org-publish-find-property entry :date project))))))
228:          (entry
229:           (sxml-to-xml
230:            `(li (@ (data-date ,created-at)
231:                    (class ,(mapconcat (lambda (tag) tag) roam-tags " ")))
232:                 (span (@ (class "sitemap-entry-date")) ,created-at)
233:                 (a (@ (href ,(file-name-sans-extension entry)))
234:                    ,(org-publish-find-title entry project))
235: 
236:                 ,(if roam-tags
237:                      `(span (@ (class "sitemap-entry-tags"))
238:                             ,(concat "("
239:                                      (mapconcat (lambda (tag) tag) roam-tags ", ")
240:                                      ")")))))))
241:         (list entry roam-tags)))

# Sitemap page

From the function above, the roam-tags are placed into a flattened list, duplicate values removed and sorted alphabetical ascending. These are turned into tags on the page used for filtering the entries by topic. All of the JS used for filtering is provided by the UIkit CSS framework.

242: (defun my/sitemap (title list)
243:   (let* ((unique-tags
244:           (sort
245:           (delete-dups
246:             (flatten-tree
247:               (mapcar (lambda (item) (cdr (car item)))
248:                       (cdr list))))
249:           (lambda (a b) (string< a b)))))
250:     (concat
251:     "#+TITLE: " title "\n\n"
252:     "#+BEGIN_EXPORT html\n\n"
253:     (sxml-to-xml
254:      `(div (@ (id "tag-filter-component")
255:               (uk-filter "target: .js-filter"))
256:            (div (@ (class "tags uk-subnav uk-subnav-pill"))
257:                 (span (@ (uk-filter-control "group: tag"))
258:                       (a (@ (href "#")) "ALL"))
259:                 ,(mapconcat (lambda (item)
260:                               (format "<span id=\"%s\" uk-filter-control=\"filter: .%s; group: tag\"><a href=\"#\">%s</a></span>"
261:                                       (concat "filter-" item)
262:                                       item
263:                                       item))
264:                             unique-tags
265:                             "\n"))
266:            (ul (@ (class "uk-subnav uk-subnav-pill"))
267:                (li (@ (uk-filter-control "sort: data-date; group: date"))
268:                    (a (@ (href "#")) "Ascending"))
269:                (li (@ (uk-filter-control "sort: data-date; order: desc; group: date")
270:                       (class "uk-active"))
271:                    (a (@ (href "#")) "Descending")))
272:            (ul (@ (class "sitemap-entries uk-list uk-list-emphasis js-filter"))
273:                ,(mapconcat (lambda (item) (car (car item)))
274:                           (cdr list)
275:                           "\n"))))
276:     "\n#+END_EXPORT\n")))

# Derived backend

You can derive a custom backend from an existing one and can override certain functions. In this example, my-site-html derives from html and overrides template, link, and headline functions.

277: (org-export-define-derived-backend
278:     'site-html
279:     'html
280:   :translate-alist
281:   '((template . my/org-html-template)
282:     (link . my/org-html-link)
283:     (headline . my/org-html-headline))
284:   :options-alist
285:   '((:page-type "PAGE-TYPE" nil nil t)
286:     (:html-use-infojs nil nil nil)
287:     (:updated "UPDATED" nil nil t)
288:     (:roam_tags "ROAM_TAGS" nil nil split)))

# Publishing

# Output paths

This is a helper function that converts an org-mode file name to a directory of the same name, downcased and without the filename extension. So if the filename is my-post.org, a sub-directory would be created in the publishing directory called my-post/. The sitemap is indented to be at the root of the publishing directory (ie, the homepage). This function is called in the next code block.

289: (defun get-article-output-path (org-file pub-dir)
290:   (let ((article-dir (concat pub-dir
291:                             (downcase
292:                               (file-name-as-directory
293:                               (file-name-sans-extension
294:                                 (file-name-nondirectory org-file)))))))
295:     (if (string-match "\\/sitemap.org$" org-file)
296:         pub-dir
297:         (progn
298:           (unless (file-directory-p article-dir)
299:             (make-directory article-dir t))
300:           article-dir))
301:     ))

# The publishing function (and conditional TOCs)

This function does a few things:

  • It adds the export option to generate a table of contents only if there are more than 3 headlines. Otherwise, I don’t see a point to rendering a TOC.
  • Next it calls the helper function above to create the output directory and appends index.html to the result. This ends up being the article-path for a post. For example, if the filename is my-post.org, the article path would be /my-post/index.html.
  • Finally, it calls org-publish-org-to which publishes a file using the selected backend.
302: (defun my/org-html-publish-to-html (plist filename pub-dir)
303:   (with-current-buffer (find-file filename)
304:     (when (> (length (org-map-entries t)) 3)
305:       (insert "#+OPTIONS: toc:t\n")))
306:   (let ((article-path (get-article-output-path filename pub-dir)))
307:     (cl-letf (((symbol-function 'org-export-output-file-name)
308:               (lambda (extension &optional subtreep pub-dir)
309:                 (concat article-path "index" extension))))
310:       (org-publish-org-to 'site-html
311:                           filename
312:                           (concat "." (or (plist-get plist :html-extension) "html"))
313:                           plist
314:                           article-path))))
315: 

# The project alist

This is the configuration for the publishable projects. Each project can be published independently with org-publish and the project name (eg (org-publish "site")), or all of them with org-publish-all.

316: (setq org-publish-project-alist
317:       (list
318:        (list "notes.alex-miller.co"
319:              :base-extension "org"
320:              :base-directory "./"
321:              :publishing-function '(my/org-html-publish-to-html)
322:              :publishing-directory "./public"
323:              :auto-sitemap t
324:              :sitemap-function 'my/sitemap
325:              :sitemap-title my/sitemap-title
326:              :sitemap-format-entry 'my/sitemap-format-entry
327:              :sitemap-sort-files 'alphabetically
328:              :with-title nil
329:              :with-toc nil)
330:        (list "images"
331:              :base-extension "png\\|jpg\\|svg"
332:              :base-directory "./images"
333:              :publishing-directory "./public/images"
334:              :publishing-function 'org-publish-attachment)
335:        (list "site" :components '("notes.alex-miller.co" "images"))))

# notes.alex-miller.co

This publishes the org-mode files. I keep them in the root directory. I have a few other folders for other note types that I don’t publish. The HTML output is placed in the ./public directory which is gitignored. The sitemap functions are documented above. TOCs are only generated for notes that have more than 3 headlines.

# images

I sometimes link and display images in my org-notes, like plantuml renderings. I put these in the ./images directory. This basically just copies them over to the /images directory of the site. This ensure that links and/or inline images work. (See this emacs.stackexchange answer for where I got the idea).

# site

It contains everything needed to build the site.