So, inspired by a recent HackerNews post about somebody’s TIL’s for five years (found here), I want to start writing about things I learn about each month. I feel like this format is easier to consume (I don’t think anyone will read a wall of text in a Git repo), but this will also help me organize my thoughts. Writing things down helps me remember.
I’ll also highlight something that I didn’t necessarily learn, but re-discovered due to how helpful they are.
AWS
Boto3 uploading to S3 (5/12/20)
So as I was working on our website for Kudzera, LLC (which I’m statically hosting in AWS), I was trying to set up CI through GitHub Actions. Part of that was automatically uploading to S3. However, when I did, the webpage didn’t render (it instead prompted for download). Turns out, Boto3 uploads as a binary/octet-stream content type. With some use of mimetypes in Python, we can set the content type on upload
mimetype = mimetypes.guess_type(file_name)[0]
content_type = mimetype if mimetype is not None else "binary/octet-stream"
response = s3_client.upload_file(file_name,
BUCKET_NAME,
object_name,
ExtraArgs={
'ContentType':
content_type
})
Git/GitHub
Finding commits that are in one branch but not others (5/4/20)
git log old-branch ^new-branch
That one little ^ was a cool little feature I never knew about. I was trying to compare branches with a diff (one old and one new), as it turned out we had missed merging some commits in our devel branch when putting a feature back in an old release. A diff was useless, as there were legitimate things removed and added in the new branch. This one command gave me everything I needed (you may want to pass –no-merges as well)
Running GitHub Actions only on a single branch
You may not want your GitHub action to happen on development branches, so if that’s the case you can do:
name: CI
on:
push:
branches:
- master
Linux
Copying to clipboard (5/4/20)
xclip -selection clip-board <filename>
This is something I knew about, but never used. I copy my SSH key out to a GUI when launching VMs in a cloud all the time, and always hated having to do it. Now I have a convenient alias to the command above.
Pandas
Checking if an Element is in Pandas Series (5/20/20)
So doing a quick pandas script for some data analysis had me encounter a strange behavior. I was using pandas.Series and I wanted to check to see if there was an element inside that series. Thinking that a pandas.Series was a subtype (semantically speaking) of a list, but it turns out that if you do element in series , it just checks the index, not necessarily the values. Instead you should be doing element in series.values.
Python
Beware the Pip Cache (5/29/20)
I ran into an issue with tox the other day, where an import of a library in my unit tests ran into an error finding libffi.so.6. No biggie, I had upgraded my OS since last time I worked on this package. So I cleared the .tox directory and let it pull things down. Same problem.
My system python and my pyenv python did not show the problem, so I know that it wasn’t an OS problem (I had libffi.so.7 installed, and those python’s were using it correctly). So what was happening?
My package had cffi pinned to an older version. My system python and pyenv were using a newer version of cffi. When tox tried to pull cffi, it was hitting the pip cache (and not actually downloading and recompiling the cffi wheel.), which meant it was grabbing an older pre-compiled incompatible version.
Web Development Tools
Searching through network logs in Firefox Web Console (5/8/20)
In Firefox (and I’m sure others), when you are doing network logging, you can search through all the traffic right in the web browser. Just click the search icon and type in your criteria – it’s way easier to filter network requests than saving out traffic and searching in something like WireShark
Pat Viafore 