Guus Bosman

I'm mostly done with the Natural Language Processing specialization and tonight I was playing around with a fun proof of concept written by Andrej Karpathy.

The proof of concept is a tiny transformer architecture, "GPT mini". Unlike its big GPT-3 and GPT-4 brothers, this tiny model is character based, not word based. The text I used are three short stories by author Nescio -- which, in fact, constitute his complete works, around 200k characters. I trained it for 20,000 iterations.

I've completed the first three courses of the Natural Language Processing specialization at Coursera, and started the fourth one today. I really enjoyed the the courses so far but this fourth course is the best part: learning about Attention.

The basic concept of this was introduced in 2014, three years later the famous "Attention is all you need" paper came out.

With the rise of Large Language Models, I decided to dive into machine learning again and catch up on how these LLMs truly work. I'm following a course by Deeplearning.ai on Natural Language Processing.

I'm at the end of the second course and we're going through Continuous Bag of Words, which was part of the original word2vec algorithm that was so influential. It's really fun. The previous week was about n-grams, which was also nice to program with.

I successfully completed a certification for Azure, the cloud offering from Microsoft.

I took the exam in our basement; it was strange to be on video all the time. But the process was nice and smooth.

A script I wrote a few years ago to help me with pictures on the website stopped working... It assumed every year started with 201, haha.

FoundPos := RegExMatch(clipboard, "^201[0-9][0-9][0-9][0-9][0-9].*\.[jJ][pP][gG]", FoundFileName)

I moved my blog to EC2 last weekend and last night the instance stopped working for the first time.

"Design for failure", is the mantra, and I'll do some work on automated reboots when monitoring fails.

Seems like the httpd server went ran out of memory last night.

Dec 15 00:26:49 ip-172-31-60-46 kernel: Out of memory: Kill process 8250 (httpd) score 37 or sacrifice child
Dec 15 00:26:49 ip-172-31-60-46 kernel: Killed process 8250 (httpd) total-vm:526616kB,...
Dec 15 00:26:53 ip-172-31-60-46 kernel: httpd invoked oom-killer...

I just completed the exam for AWS Certified Cloud Practitioner, a certificate that shows I have basic knowledge about the different Amazon Web Service offerings.

The exam was relatively easy. I've been preparing for this conference for a while, and done quite a bit of studying on the various AWS products. I did study a few trial exams from Udemy.com, and those were helpful for questions on more basic stuff like the Billing and Support models -- the less technical topics.

Still, it's been a while since I had done an official test and I was a little nervous, and now I'm very happy I passed the test. "Certified individuals" also get access to a special area here at the re:Invent 2018 conference.

Hmmm, I'm not very proud of this but it looks like my site has had an open redirect for a good while. Years, in fact.

It's a custom script that I wrote 15 years ago and somehow survived the migration from the various CMS that I've used.

The original script had a reference to Php-Nuke from 2002. I've now removed it, finally.

Today I switched from Firefox to Chrome.

I've been using Firefox since the mid 2000's, when it came out to replace the bulky Mozilla suite. It's with some nostalgia that I'm making the switch, but I've ran into several bugs in Firefox that weren't getting resolved.

(While typing this, I did discover that Chrome does not have auto-recover for text areas, which is annoying but thankfully there is a browser extension Typio that helps).

The nail in the coffin was the release of Firefox Quantum last year, when they stopped supporting XUL plugins. I understand the rationale -- but it broke several important plugins, including all those for mass-password reset. In the six months since the switch there haven't been any new plugins written that allow me to change all my 100+ work logins at once.

Firefox also did not work when I was presenting something under WebEx. That might have been WebEx's problem -- but it's annoying enough since I screen-share at least once a day.

So here we are, in a brave new world,

For our college recruitment I gave a tech talk at Carnegie Mellon University last week. It was fun talking with students. They were younger than I remember. It seems like my talk went over quite well, and at the end I gave them my three pieces of advice for looking for jobs: 1. Work where the firm makes its money, 2. Your supervisor is very important. When looking for a job, try to interview her/him as much as possible, 3. Do the hard things. In college: take the hard courses. At work: investigate the hard problems. Be the go-to-person.

The next morning I went for a run downtown, crossing all three rivers. Pittsburgh is a beautiful city.

I drove to Pittsburgh from Arlington -- about 4 hours clean driving time.