Micro.Blog and Hugo

 29. May 2018
/resources/images/teaserpics/micro_hudf3b0686da53bdfae061dce3e4c3f8fc_5929_300x0_resize_q75_box.jpg

Introduction

No matter what I tried I could not get Micro.blog posts to go to to my Hugo site. This is my hack to make it work.I set up some scripts to run on my machine whenever I compile the site.

Caveats and Setup Needed

Setups

GitHub

A. Create an empty repo. I named mine “Micro.Blog”. B. Generate a GitHub access token. C. Go to Settings

GitHub Srttings

D. Developer Settings

Dev Srttings

E. Personal Access Token

Personal Access

F. Generate Access Token

Personal Access

G. GitHub steps complete.

Micro.Blog Settings

A. Go to Account B. Click Edit Domain & Design

Edit Domain & Design

C. Click Edit

Edit

D. Plug in GitHub repo name and your Personal Access Key

Personal Access Key

E. Click Update Microblog Settings

Hugo Setup

A. I am using git submodules to link my various repos together. B. Create a micro.blog submodule

git submodule add https://github.com/~your account~/~your micro.blog repo~.git micro.blog

C. I added a second submodule for my site deploy scripts

git submodule add https://github.com/~your account~/~your deploy scripts repo~.git deploy

D. Add submodule folder to the ignoreFiles list in config.toml

ignoreFiles = [ "deploy/", "micro.blog/" ]

E. Run the deploy scripts

Deploy Scripts

Python

I am using one of the Jekyll to Hugo scripts found here. I like Convert to Hugo more, simply because I could read the code easier. I edited the front matter section to fit my setup.

I use the command:

python ./deploy/ConvertToHugo.py ./micro.blog/_posts ./content/microblog

input path: ./micro.blog/_posts output path: ./content/microblog

#!/usr/bin/env python
# coding:utf-8

import os
import re
import yaml
from datetime import datetime
import argparse

__author__ = 'coderzh'


content_regex = re.compile(r'---([\s\S]*?)---([\s\S]*)')


def convert_front_matter(front_data, post_date, url):
    if post_date:
        front_data['date'] = post_date.strftime('%Y-%m-%d')
    front_data['url'] = url
    front_data['categories'] = 'micro.blog'
    front_data['resImgTeaser'] =  'teaserpics/micro.jpg'
    del front_data['layout']

    for tag in ['tags', 'categories', 'category']:
        if tag in front_data and isinstance(front_data[tag], basestring):
            front_data[tag] = front_data[tag].split(' ')
    

replace_regex_list = [
#    (re.compile(r'^```(.*?)\n(.*?)\n```', re.DOTALL), r'
\n\2\n
'),
(re.compile(r'<!--\smore\s-->'), ''), (re.compile(r'\{%\sraw\s%\}(.*)\{%\sendraw\s%\}'), r'\1') ] def convert_body_text(body_text): result = body_text for regex, replace_with in replace_regex_list: result = regex.sub(replace_with, result) return result def write_out_file(front_data, body_text, out_file_path): out_lines = ['---'] front_string = yaml.dump(front_data, width=1000, default_flow_style=False, allow_unicode=True) out_lines.extend(front_string.splitlines()) out_lines.append('---') out_lines.extend(body_text.splitlines()) with open(out_file_path, 'w') as f: f.write('\n'.join(out_lines)) filename_regex = re.compile(r'(\d+-\d+-\d+)-(.*)') def parse_from_filename(filename): slug = os.path.splitext(filename)[0] m = filename_regex.match(slug) if m: slug = m.group(2) post_date = datetime.strptime(m.group(1), '%Y-%m-%d') return post_date, '/%s/%s' % (post_date.strftime('%Y/%m/%d'), slug) return None, '/' + slug def convert_post(file_path, out_dir): filename = os.path.basename(file_path) post_date, url = parse_from_filename(filename) content = '' with open(file_path, 'r') as f: content = f.read() m = content_regex.match(content) if not m: print 'Error match content: %s' % file_path return False front_data = yaml.load(m.group(1)) if not front_data: print 'Error load yaml: %s' % file_path return False ''' if 'layout' in front_data: if post_date: out_dir = os.path.join(out_dir, front_data['layout'], str(post_date.year)) else: out_dir = os.path.join(out_dir, front_data['layout']) ''' if not os.path.exists(out_dir): os.makedirs(out_dir) out_file_path = os.path.join(out_dir, filename) convert_front_matter(front_data, post_date, url) body_text = convert_body_text(m.group(2)) write_out_file(front_data, body_text, out_file_path) return True def convert(src_dir, out_dir): count = 0 error = 0 for root, dirs, files in os.walk(src_dir): for filename in files: try: if os.path.splitext(filename)[1] != '.md' or filename in ['README.md', 'LICENSE.md']: continue file_path = os.path.join(root, filename) common_prefix = os.path.commonprefix([src_dir, file_path]) rel_path = os.path.relpath(os.path.dirname(file_path), common_prefix) real_out_dir = os.path.join(out_dir, rel_path) if convert_post(file_path, real_out_dir): print 'Converted: %s' % file_path count += 1 else: error += 1 except Exception as e: error += 1 print 'Error convert: %s \nException: %s' % (file_path, e) print '--------\n%d file converted! %s' % (count, 'Error count: %d' % error if error > 0 else 'Congratulation!!!') if __name__ == '__main__': parser = argparse.ArgumentParser(description='Convert Jekyll blog to GoHugo') parser.add_argument('src_dir', help='jekyll post dir') parser.add_argument('out_dir', help='hugo root path') args = parser.parse_args() convert(os.path.abspath(args.src_dir), os.path.abspath(args.out_dir))

Shell

The following is my deploy script which compiles, commits, and publishes my site.

I use the command to run it:

./deploy/deploy.sh "Commit Message"
#!/bin/bash

echo -e "\033[0;32mDeploying updates to MinimalWorkflow.com...\033[0m"

#git remote set-url origin git@github.com:wilsongis/mwf.git
#
#Get Micro.Blog Changes

echo -e "\033[0;32mGetting Micro.Blog Changes...\033[0m"
cd micro.blog
git pull
cd ..
python ./deploy/ConvertToHugo.py  ./micro.blog/_posts ./content/microblog
echo -e "\033[0;32mAdding MWF Changes to GitHub...\033[0m"
# Add changes to git.
git add .

# Commit changes.
msg="rebuilding site `date`"
if [ $# -eq 1 ]
  then msg="$1"
fi
git commit -m "$msg"

# Push source and build repos.
git push #origin master

# Come Back up to the Project Root

echo -e "\033[0;32mRSync Public to Host..\033[0m"
USER=<your username>
HOST=<your host>
DIR=<host directory>  

rm -R public

hugo && rsync -avz -e 'ssh -p 2233' --delete public/ ${USER}@${HOST}:~/${DIR}

exit 0