Compiling Cub for Shortcuts

Earlier this week I heard about Shortcuts JS, which uses JavaScript to generate Shortcuts. Here’s an example from their homepage:

// We'll use this later to reference the output of a calculation
let calcVar = actionOutput();

// Define a list of actions
const actions = [
  comment({
    text: 'Hello, world!',
  }),
  number({
    number: 42,
  }),
  calculate({
    operand: 3,
    operation: '/',
  }, calcVar),
  showResult({
    // Use the Magic Variable
    text: withVariables`Total is ${calcVar}!`,
  }),
];

While this is a good first step to writing “real code” to make Shortcuts, specifying the operands and others in this fashion is clunky. I wondered how easy it would be to use the syntax tree instead to create the Shortcut, and Will Richardson has done that exact thing for Cub in a blog post:

All I have to do was traverse the syntax tree and generate a Shortcuts file. In terms of code this is fairly straightforward - just add an extension to each AST node that generates the corresponding code.

I’m not familiar enough with iOS app development or Swift to do it myself, but it would be really interesting to write an app that can use something like swift-ast to generate Shortcuts. Who knows what power iOS users could get if they could program advanced Shortcuts using Swift?

State of the Apps 2018

Inspired by Cortex’s annual State of the Apps discussion, I thought it would be fun to start documenting what I’m using the most on my phone every year. Below are the my most-used 3rd party apps of the year.

Productivity

1Password
I moved from LastPass to 1Password earlier this year and I couldn’t be happier. With the Password AutoFill API, 1Password integrates with the iOS keyboard to fill in logins with only one tap, and then app even copies one-time authentication codes to the clipboard. 1Password’s integration with iOS 12 has even stellar and I can’t recommend it more.

Todoist
A cross-platform task management system, Todoist is how I keep track of all of my projects, from TA grading deadlines to senior design final deliverables. While Todoist’s UI doesn’t have a native app feel, it’s clean and consistent.

Slack
Aptly categorized by Federico Viticci as a barely passable iOS client designed to access a web app, Slack was the app I used almost every day to communicate with several groups at my university. Slack supports some newer iOS features such as grouped notifications, but the UI mostly resembles its desktop Electron client and often hides away features in non-obvious places, such as touching and holding a message to add a reaction.

Social & Entertainment

Instagram
While I’m not a fan of the company behind Instagram for many many many reasons, Instagram is a place where I can keep up with my friends via Stories and posts, and I’ve found it to be much more positive overall than other social networks this year.

Apollo
Reddit is one of my primary social networks I use every day, and Apollo is hands down my favorite way to experience Reddit on any platform. The design is intuitive and easy to personalize, and it has a fantastic night mode. I can’t stop recommending the app to my friends that use the first party client.

lire
A few months ago I was using IFTTT applets to monitor RSS feeds and push them to a specific project in Todoist, but it because untenable after adding too many feeds. I moved to Inoreader as my RSS service and lire as my RSS reader. The app has a clean, native-looking design, and it uses its own extractor to display a story’s full text. I wish the per-feed options were more straightforward and easy to access, but overall I’m very happy using it as my primary RSS reader.

Overcast
Overcast has been my podcast player go-to app since I started using it over 3 years ago. Smart Speed and Voice Boost are still industry-leading features that I can’t live without, and new features are being continuously added, such as full text search in version 5.0.

GroupMe
Another app in the “barely passable” category, I use GroupMe every day at my university for group chats ranging from friend groups to club event announcements. The app is missing many basic features I expect from a communication app, including read-message syncing across devices and platforms, and I cannot wait until I can delete this app from my phone.

Health & Finance

AutoSleep
While the Apple Health app tracks basic sleep information, AutoSleep provides in-depth detail for sleep trends and day-to-day stats. I check it often to look at my readiness for the day, cumulative sleep debt, and overall sleep time consistency. It was especially interesting to compare one particularly-gruesome senior design week of little sleep with the rest of the semester averages.

YNAB
While I had toyed with budgeting on Mint in the past, YNAB (or You Need a Budget) is a great way to manage your savings and expenses. The service requires a subscription, but its features such as Bank Syncing and Goal Tracking as well as it’s straightforward usage make it an excellent deal. YNAB has given me a clear way to know exactly what I’m spending every month.

Venmo
A digital wallet app owned by PayPal, Venmo allows me to send and receive payments from other people. Similar apps like Apple Pay and Cash App are available, but Venmo is what nearly all of the people in my social circle have centralized on. Further, with the addition of a Venmo card option released in the summer, Venmo has made it to handle group events and easily split payments.

Utilities

Deliveries
This app has allowed me to neurotically check my packages from every online retailer. All I need to do is copy the tracking number from USPS, FedEx, or UPS and paste it into Deliveries, and then I can push updates whenever the package status has changed. Deliveries is so good it encourages me to buy more things from Amazon, if that’s even possible.

CARROT Weather
I switched from Dark Sky to CARROT Weather in February, and that was mostly because of the fun sadistic messages that managed to be both funny and relevant. At the same time, the app has seen numerous updates this year to support multiple weather locations, highly customizable Apple Watch complications, and more.

1Blocker X
For any website that doesn’t respect its users and decides to use popups, newsletter signup prompts, and auto-playing video ads, 1Blocker X is great at preventing them in the background. The app does what a utility should do: work without me even remembering I have a content blocker turned on.

Google Maps
Whenever I need to travel more than 10 minutes, I turn on Google Maps for live traffic updates and possible shorter route recommendations. In addition to retailer busy times and reviews, Google Maps is a great way to gather details about locations around me.

Due
For smaller tasks that I want to be reminded of over and over, I put them in Due, which is great at spamming me with notifications until I mark the task complete. Plus, the app recently added custom snooze times from a notification in a recent update.

Google Photos
I use Google Photos both as a secondary back up to iCloud Photos and for more power photo analytics than what Apple provides. For now, I’ll gladly trade Google using my aggregated photo data to have an easy way to search photos by person, place, or thing across platforms. I also share albums and pictures through Google Photos to groups that have a mix of Android an iOS devices.

Tailor
A nifty utility app that stitches multiple screenshots into one vertical image, Tailor provides an easy way to send more readable conversations to other people, whether they’re from Messages, Slack, or other apps.

Home Screen

Finally, here’s a picture of my home screen at the end of 2018:

The Case of the 500-Mile Email

Trey Harris, writing to sage-members:

I was working in a job running the campus email system some years ago when I got a call from the chairman of the statistics department.

“We’re having a problem sending email out of the department.”

“What’s the problem?” I asked.

“We can’t send mail more than 500 miles,” the chairman explained.

I choked on my latte. “Come again?”

“We can’t send mail farther than 500 miles from here,” he repeated. “A little bit more, actually. Call it 520 miles. But no farther.”

While it’s an older story, it’s a fantastic one. And it’s a great reminder to check your software versions.

Web API and Templates with Python requests and Jinja2

Introduction

Web APIs are becoming an increasingly popular method to retrieve and store data, and they can be an extremely powerful tool for any programmer. In this walkthrough, you will use GroupMe’s public API to retrieve the last few messages in a group and display it using a custom HTML template in Python 3.7.1:

While we’ll be using specific GroupMe API endpoints, this walkthrough will help you to learn the basics of working with web APIs in general.

Set up

Before we begin, you need to have the following modules installed:

• requests (for connecting to the API)
• jinja2 (for adding data to our template)

Using a Web API

Working with requests

The requests library is great for creating HTTP requests, and it has fantastic documentation. We’ll be using requests.get(url) to get the information we need:

>>> import requests
>>> # Use JSONPlaceHolder as an example
>>> req = requests.get('https://jsonplaceholder.typicode.com/todos/1')
>>> print(req.text)
{
  "userId": 1,
  "id": 1,
  "title": "delectus aut autem",
  "completed": false
}

Create an Application

To use GroupMe’s API, we need to first register an application to get our API key. Log into the developer application page and click “Create Application”. You’ll be taken to this page:

We won’t be using the Callback URL, so set that to any valid URL. Fill in the developer information and submit the form, and you’ll be taken to the application’s detail page:

Copy the Access Token at the bottom of the application page. That’ll be our API key.

Find the Group ID

To get messages in a group, we’ll first need to get the group’s ID. GroupMe’s documentation says to use the base url https://api.groupme.com/v3 and has the following example using curl:

$ curl -X POST -H "Content-Type: application/json" -d '{"name": "Family"}' https://api.groupme.com/v3/groups?token=YOUR_ACCESS_TOKEN

From this, we know that the url we use with requests.get() will be in the form¹

https://api.groupme.com/v3/...?token=YOUR_ACCESS_TOKEN

Looking at the groups API, we can use GET /groups to retrieve a list of our most recent groups:

>>> import requests
>>> API_KEY = 'YOUR_API_KEY'
>>> base_url = 'https://api.groupme.com/v3'
>>> req = requests.get(f'{base_url}/groups?token={API_KEY}')
>>> req.content
b'{"response":[{"id":"1111","group_id":"2222","name":"My Group Name","phone_number":"+1 5555555555","type":"private","description":"Group description goes here",
...

First, we construct the url for the API request and pass it as the argument to requests.get(). Then, we print the result of the API request stored as req.content.

The response we get from GroupMe is a JSON-formatted string, so we’ll move our script into its own file and parse the string using Python’s standard json library:

import json
import requests

API_KEY = 'YOUR_API_KEY'
BASE_URL = 'https://api.groupme.com/v3'

def api_request(request):
    '''Returns the data from a request (eg /groups)'''
    url = f'{BASE_URL}{request}?token={API_KEY}'
    req = requests.get(url)

    if not req.content:
        return None

    # We only want the data associated with the "response" key
    return json.loads(req.content)['response']

if __name__ == '__main__':
    groups = api_request('/groups')
    print(len(groups), 'group(s)')
    for group in groups:
        print(f'ID {group["group_id"]}: {group["name"]}')

The function api_request does the work of creating the final URL string for us. Then, it makes the request and checks that something was returned by GroupMe’s servers. If something was sent back to us, the content is converted² from a string into a Python object using json.loads(). Finally, we return the data associated with the key response, because the rest is metadata unimportant to us.

When we run the script, our most recent groups are returned (as a JSON object decoded into a Python object). The result will tell us the group names and their group IDs:

3 group(s)
ID 11111111: Python Tips and Tricks
ID 22222222: University Friend Group
ID 33333333: GitHub Chat

Get Messages for a Group

We have a list of our group IDs, so we can use the following API to get a list of recent messages for one group:

GET /groups/<group_id>/messages

Let’s add this endpoint to our script as get_messages_for_group(group_id):

import json
import requests

API_KEY = 'YOUR_API_KEY'
BASE_URL = 'https://api.groupme.com/v3'

def api_request(request):
    '''Returns the data from a request (eg /groups)'''
    url = f'{BASE_URL}{request}?token={API_KEY}'
    req = requests.get(url)

    if not req.content:
        return None

    # We only want the data associated with the "response" key
    return json.loads(req.content)['response']

def get_messages_for_group(group_id):
    response = api_request(f'/groups/{group_id}/messages')

    # Just return the messages (and none of the metadata)
    return response['messages']

if __name__ == '__main__':
    messages = get_messages_for_group(YOUR_GROUP_ID)
    print(messages[0])

Our script will get the messages for a group (fill in YOUR_GROUP_ID) and print the most recent one. Running it will print something like:

{'attachments': [], 'avatar_url': None, 'created_at': 1544810700, 'favorited_by': [], 'group_id': '11112233', 'id': '882882828288288282', 'name': 'Johnny Test', 'sender_id': '22558899', 'sender_type': 'user', 'source_guid': 'android-11111111-3eee-4444-9999-aaaabbbbcccc', 'system': False, 'text': "Hello everyone!", 'user_id': '55551111'}

We can see from the message’s data that the sender’s name “Jonny Test” and the text was “Hello everyone!” Next, we should organize our API results as Python objects to be easier to expand on.

Creating Classes for API Objects

Now that we’re ready to start processing the data from the API, it’s a good time to create objects to represent our API objects. With Python classes, we can keep only the data we need and begin to process our own information. We’ll initialize our API objects by passing them the decoded Python object from api_request(request). This way, we can more easily add class properties without needing to change our request function.

Let’s make two classes, Group and Message:

class Message:
    def __init__(self, json):
        self.user_id = json['user_id']
        self.name = json['name']
        self.text = json['text']

class Group:
    def __init__(self, json):
        self.id = json['group_id']
        self.name = json['name']
        self.messages = []

Then we can add a method to Group to fetch its recent messages:

def get_recent_messages(self):
    messages = get_messages_for_group(self.id)

    # Convert each message to our object
    for message in messages:
        new_message_object = Message(message)
        self.messages.append(new_message_object)

And then we can use our script to print out the messages for a group:

import json
import requests

API_KEY = 'YOUR_API_KEY'
BASE_URL = 'https://api.groupme.com/v3'

def api_request(request):
    '''Returns the data from a request (eg /groups)'''
    url = f'{BASE_URL}{request}?token={API_KEY}'
    req = requests.get(url)

    if not req.content:
        return None

    # We only want the data associated with the "response" key
    return json.loads(req.content)['response']

def get_messages_for_group(group_id):
    response = api_request(f'/groups/{group_id}/messages')

    # Just return the messages and none of the metadata
    return response['messages']

class Message:
    def __init__(self, json):
        self.user_id = json['user_id']
        self.name = json['name']
        self.text = json['text']

class Group:
    def __init__(self, json):
        self.id = json['group_id']
        self.name = json['name']
        self.messages = []
        self.get_recent_messages()

    def get_recent_messages(self):
        messages = get_messages_for_group(self.id)

        # Convert each message to our object
        for message in messages:
            new_message_object = Message(message)
            self.messages.append(new_message_object)

if __name__ == '__main__':
    groups_json = api_request('/groups')
    my_group = Group(groups_json[0])

    for message in my_group.messages:
        print(message.text)
        print(f'-- {message.name}')
        print()

The result is the most recent messages for our most recent group:

Hello everyone!
-- Johnny Test

Hi guys I had a question about using @classmethod
-- Alexa Jones

Wow great work!
-- Katie Alendra

We have the data in a manageable format, so it’s time to start formatting it in a readable form.

Using Jinja Templates

We’ve come a long way so far! First, we learned how to make HTTP GET requests to a server. Then, we used GroupMe’s API docs to fetch data about different groups and messages, and then we created Python classes to better organize our information. Let’s create a Jinja template to print out our data.

Create the Template

First, I’ll make a group.html file that has the framework of I want the web page to look like:

<body>
    <h1>GROUP NAME</h1>
    <br />

    <!-- Repeat for every message -->
    <p><b>MESSAGE CONTENT</b> &mdash; NAME</p>
</body>

With Jinja, variables are inserted into the template using {{ variable_name }}, and logic statements have a form such as:

{% if should_display %}
    <p>This message should be displayed</p>
{% endif %}

If we assume that we’ll pass a Group() instance into our Jinja template with the variable name group, we can rewrite group.html as:

<body>
    <h1>{{ group.name }}</h1>
    <br />

    <!-- Repeat for every message -->
    {% for message in group.messages %}
    <p><b>{{ message.text }}</b> &mdash;{{ message.name }}</p>
    {% endfor %}
</body>

Note the {% endif %} and {% endfor %} in the above snippets; they’re required for all conditionals and loops.

Populate the Template

With our template written, let’s go back to our script and add a section to import our template using jinja2.

with open('group.html', 'r') as f:
    contents = f.read()

template = jinja2.Template(contents)
filled_template = template.render(group=my_group)

with open('output.html', 'w') as f:
    f.write(filled_template)

First, we read the contents of our template file. Because we’re only going to use one file, we can just load the text of our template into jinja2.Template, and then we can render the template by passing our my_group variable (from our main script) as group. Finally, we write the contents to output.html to view it in a browser.

Now we have our full script:

import json
import jinja2
import requests

API_KEY = 'YOUR_API_KEY'
BASE_URL = 'https://api.groupme.com/v3'

def api_request(request):
    '''Returns the data from a request (eg /groups)'''
    url = f'{BASE_URL}{request}?token={API_KEY}'
    req = requests.get(url)

    if not req.content:
        return None

    # We only want the data associated with the "response" key
    return json.loads(req.content)['response']

def get_messages_for_group(group_id):
    response = api_request(f'/groups/{group_id}/messages')

    # Just return the messages and none of the metadata
    return response['messages']

class Message:
    def __init__(self, json):
        self.user_id = json['user_id']
        self.name = json['name']
        self.text = json['text']

class Group:
    def __init__(self, json):
        self.id = json['group_id']
        self.name = json['name']
        self.messages = []
        self.get_recent_messages()

    def get_recent_messages(self):
        messages = get_messages_for_group(self.id)

        # Convert each message to our object
        for message in messages:
            new_message_object = Message(message)
            self.messages.append(new_message_object)

if __name__ == '__main__':
    groups_json = api_request('/groups')
    my_group = Group(groups_json[0])

    with open('group.html', 'r') as f:
        contents = f.read()

    template = jinja2.Template(contents)
    filled_template = template.render(group=my_group)

    with open('output.html', 'w') as f:
        f.write(filled_template)

Once run, we can view our output.html in a browser:

<body>
    <h1>Python Tips and Tricks</h1>
    <br />

    <!-- Repeat for every message -->
    <p><b>Hello everyone!</b> &mdash;Johnny Test</p>
    <p><b>Hi guys I had a question about using @classmethod</b> &mdash;Alexa Jones</p>
    <p><b>Wow great work!</b> &mdash;Katie Alendra</p>
</body>

Conclusion

We’ve walked through how to access and parse a web API using the requests library, how to represent and organize the API data using Python classes, and how to render the information in a custom format using a Jinja template. Now go create your own cool stuff using APIs!

Expand Tilde Paths in Bash and Python

Sometimes it’s necessary to reference files in a script using ~. For example, if you want to schedule a cron job to run a script in a folder and place the results in the same folder, it’s helpful to use absolute referencing of the files in the script.

Bash

Here’s my first attempt to append to a file:

$ ./my_folder/run.sh >> "~/my_folder/output.txt"
-bash: ~/my_folder/output.txt: No such file or directory

The issue with the above line is that the ~ is not expanded to the home directory (such as /home/username/) because it is inside the quotes. To fix this, move the path outside of the quotes, but leave the filename in single quotes (to escape the . in the extension):

$ ./my_folder/run.sh >> ~/my_folder/'output.txt'

Python

I encountered a similar issue in Python:

>>> with open('~/my_folder/output.txt', 'r') as f:
...   contents = f.read()
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '~/my_folder/output.txt'

This can be fixed using os.path.expanduser(path):

>>> import os
>>> filename = os.path.expanduser('~/my_folder/output.txt')
>>> with open(filename, 'r') as f:
...   contents = f.read()
...
>>> print(contents)
10

iPad Screenshots

Dr. Drang:

On the iPad, ⇧⌘3 captures the whole screen, just like the Mac (and just like capturing with the top and volume up buttons). The ⇧⌘4 shortcut also captures the whole screen, but in a neat analogy to the Mac, it immediately puts you into editing mode so you can crop the capture down to a smaller size.

I don’t find these keyboard shortcuts surprising, but it is surprising that I never thought to try it on an iPad. With the new screenshot tool in macOS Mojave, I wonder what other features will reach parity on macOS and iOS in the future.

Removing Local Git Branches That Aren't `master`

Every so often, I’ll want to delete all of my local branches for a repository that aren’t the master branch. An easy command to do this is:

$ git branch | grep -v "master" | xargs git branch -d

(If you want to keep multiple branches, such as master and develop, you can chain them together using grep -v "master\|develop")

git branch lists all of the local branches for the repo, grep -v prints all of the lines from the previous command that don’t match “master”, and xargs takes each line from the previous output and runs git branch -d <output_line>.

I recommend using -d rather than -D in case git recommends not deleting the branch.

How To Change a Git Repo's Authentication Protocol

HTTPS to SSH Key

Often I need to change a git repository to use an SSH key instead of my username and password to authenticate with the remote server. In order to do so, type the following in the repository’s folder on your machine:

$ git config remote.origin.url git@github.com:username/repository_name.git

(Make sure to include the .git at the end of the repository name.)

SSH Key to HTTPS

In order to change it to do the reverse, type:

$ git config remote.origin.url https://github.com/username/repository

Auto Login for PuTTY (Windows)

Often I find myself wanting to have an easy way to SSH into a server on a Windows PC. Unfortunately, SSH keys on Windows can often be a challenge, but there’s an easy way to have PuTTY connect without needing to type in a password every time.

To create a shortcut for a PuTTY connection to automatically log in, you only need two things: the name of the profile (in PuTTY) that has the connection and appearance settings, and the password to your account (for the server). Right click on the desktop to create a new shortcut, then for the link type:

"C:\Program Files (x86)\PuTTY\putty.exe" -load "<profileName>" -pw "<password>"

If you saved PuTTY to a different location other than Program Files (x86), then you’ll also need to change the location of putty.exe in the command above.

Once you’ve created the shortcut, you can pin it to the taskbar or the start menu for easy access!

These instructions were inspired by the instructions for the Purdue ECE 264 course page.

Pointing a Github Pages Repo to a Hover Domain

My blog is currently hosted using GitHub Pages—which is a great way to host your static site or blog for free—by linking it to my custom domain that I purchased through Hover. While both of these services are amazing, connecting the two required many open tabs and several waiting periods. This post will explain the steps needed to point a GitHub Pages repo to a custom domain on Hover.

Preflight Check

Before connecting GitHub Pages to a custom domain, I first updated my blog on my repository nickymarino.github.io, and checked that it was displaying properly at its default website (normally https://nickymarino.github.io).

Link the Repo to the Domain

First you need to update your repository with your custom domain. In the settings for the repo, enter the domain in the “Custom domain” in the GitHub Pages section.

A Records on Hover

The next step is to configure Hover. Find GitHub’s current list of IP addresses to create records with. Then, go to your Hover account, select your domain, and go to the DNS tab) to add to the DNS tab on Hover. At the time of writing, these are:

185.199.108.153
185.199.109.153
185.199.110.153
185.199.111.153

Then, go to your Hover account, select your domain, and go to the DNS tab. Delete any DNS records that have an “A” under “Records”.

For each IP address on GitHub’s help pages, add a DNS record. For each, the “Type” will be A, the “Hostname” will be @, and the “TTL” can be left as the default value.

It may take several hours (or up to about a day) for the changes to take effect. Take a break, get some sleep, and then come back to your domain to make sure everything’s working. Now we can enforce HTTPS!

Create HTTPS certificate

If you head back to your repo’s settings page to enforce HTTPS, you might see the following “not yet available” error:

Per GitHub’s troubleshooting page, you need to remove and then re-add your custom domain for your repository. Wait around 24 hours for the certificate to be generated, and you should be good to go!

A New Look

I’ve owned this domain (nickymarino.com) for roughly three years now, and so far I’ve only used it as a resume/portfolio site. I’ve finally found a theme that I both appreciate and can spend time modifying to fit my needs.

My goal is to start writing (and podcasting!) more often, whether it’s a technical detail I found interesting, an overcome challenge I want to record for when I encounter it again, or anything else I can think of.

I have a few ideas up my sleeve.

You can stay updated via the site feed or on Twitter.

Let's Use LaTeX!

LaTeX is a beautiful documentation system. It’s similar to Markdown¹, but has many more features and is commonly used for academic papers and other publications that require a lot of equations. In this quick how to, we cover how to install LaTeX and use Visual Studio Code as an editor/previewer.

Installing LaTeX

I recommend downloading TeX Live for Windows/Linux, and MacTex for macOS.

Setting up our editor

If you haven’t already, install Visual Studio Code and go through a tutorial. Then, we need to install our extension for LaTeX itself. Head over to LaTeX Workshop and click install.

Using LaTeX

Now that we have our editor setup, we can write our first project. All LaTeX documents have a (non-blank) file that ends with .tex, which is the “main” file that has all of the text of the document. Since LaTeX usually generates more files (such as .log, etc.) while building the document, it’s recommended that every document you want to write has its own folder.

For starters, create a file named example.tex:

\documentclass{article}
    % General document formatting
    \usepackage[margin=0.7in]{geometry}
    \usepackage[parfill]{parskip}
    \usepackage[utf8]{inputenc}

\begin{document}
% Put your text here
This is starter text.
\end{document}

Press Ctrl-Alt-B to build your project (or use the Command Palette), then Ctrl-Alt-T to view the pdf in a new tab. The end result should look like this:

Conclusion

LaTeX and VSCode are a great combination that you can use to write beautiful reports and papers. Check out a tutorial or two to realize the full experience LaTeX has to offer.

Edit: Fanny recommends another great tutorial as well.

Edit 2: Fixed a tutorial link.