I want to keep my work and personal Github accounts separate. Here’s how I set that up:
At approximately 08:45 on November 10th, 2033, there were network connectivity issues on my home network. This is the incident review.
A new release of Longhorn came out a few days ago. I tried upgrading. It did not go well. This is the incident review.
I’ve added a Raspberry Pi node to my K3s cluster, and I don’t want it to take part in Longhorn replication.
As part of my over-engineered doorbell project, I’ve added a new Raspberry Pi node to my K3s cluster, but none of the daemonsets are being scheduled onto it.
My DS416play keeps falling off the network, requiring a hard reset. I wondered whether it was possible to monitor it somehow. It turns out that it responds to SNMP queries, so here’s my attempt to export all of that into Grafana.
This afternoon, I couldn’t run kubectl get namespaces against my K3s cluster. Instead, I got an Unauthorized error.
I’ve just come off a call with a colleague and while he was sharing his screen and we were walking through some things, I noticed that his setup is missing all of the little “quality of life” tweaks that I’ve made over the years.
If you use Erlang/OTP’s provided HTTP client as follows, it hangs:
If you’re building your application inside a Docker container, and you need to pull dependencies from a git server (e.g. Github or Gitlab) using SSH, you need to forward your SSH session to the container.
I dislike Powerpoint and I can’t get on with Visio. Here’s how I make my presentations.
I wanted to get a list of all of the GitLab projects of which I’m a member. Here’s how I used glab – GitLab’s CLI
tool – to do that.
If you want to replace an Erlang module on a remote node, but you’re unable to copy files to it, and you’re unable to make RPC calls to it, but you can get a remote console, here’s what you do:
Erlang’s gen_server (and gen_statem, etc.) allow for you to pass {debug, DebugOptions} in
gen_server:start_link/4 (et al.). The documentation for it is spread over several pages, making it hard to understand
at first. How should we use it?
In a previous post, I showed how to store sys.config and
vm.args in a ConfigMap. There are some problems with that approach, so I’m going to use Kustomize to fix that.
I want to play with OpenTelemetry, so I figured I’d install Grafana’s Tempo in my cluster. I’m going to use ArgoCD to install the Helm chart.
Here’s how you can use via in a name specification when starting a gen_server (or gen_statem, etc.) to register
your process with a process registry.
You can gen_server:call in a bunch of different ways.
My K3s cluster uses MetalLB as a bare-metal “load-balancer”, and I wondered how it worked. Here’s what I found.
I installed MetalLB using helm. Upgrading is a one-liner.
Erlang configuration is typically stored in the sys.config file. What options do we have if we want to have different
settings in this file when deploying using Kubernetes?
About 3 months ago, the Erlang nodes in my cluster stopped talking to each other. This was caused by the CA certificate expiring. In that post, I asked “How do we roll the CA certificate without downtime?”. This post explores some options.
In several of the previous posts, for example Using ‘setup’, I’ve
started a server in suite_setup/0 and needed to kill it in suite_cleanup/1. I showed a simple way to do that, but
it’s not the best. Here’s a better way to do it.
I’m running Gitea and ArgoCD on my K3s cluster, for some GitOps goodness. I noticed that Gitea 1.19.0 recently came out, with some features I want to try, such as Gitea Actions. Since I’m running 1.17.4, it’s time to upgrade.
When I originally installed MetalLB, it used a ConfigMap for
setting up address ranges. Since 0.13.2, it supports configuration using Custom Resource Definitions (CRDs). I forgot to
write a blog post about that when I upgraded.
Similar to passing the result from {setup, Setup, ... to each test, you can also pass the result from {foreach,
Setup, ... to each test, but there are some differences.
The result from setup is passed to cleanup already. What if we want to pass it to each of our tests? It’s a bit
awkward, but it looks like the following:
If you want to run some setup before (and cleanup after) a list of tests, and also some setup and cleanup for each
test, you can nest setup and foreach:
By default, Erlang distribution listens on all available network interfaces. Sometimes you don’t want this. How can we fix it?
We saw that setup runs setup/cleanup before all tests in a list. If you want to run some setup/cleanup before each
test in a list, you can use foreach:
What if we want to run some setup before a set of tests (and cleanup afterwards)?
While you could specify each test with a single function, you can also use a test “instantiator” (or generator) and have it return a list.
A simple EUnit test might look like this:
Some examples of how to use Erlang’s unit-testing framework, EUnit. Starting with simple examples and getting more advanced.
Sometimes, you’re connected to another node in an Erlang cluster, and you need to inject a new or replacement module. Here’s how.
In Expired Certificates: Incident Review, I listed a future action: “Audit the cluster to see if there are any other TLS secrets that aren’t using cert-manager.” Here’s how I did it using Elixir Livebook.
During the morning of March 18th, 2023, while investigating this incident, Firefox reported a “your connection is not secure” error when connecting to my gitea server. This is the incident review.
On the morning of March 18th, 2023, accessing any sites hosted by my Kubernetes cluster would fail with a “Connection Timed Out” error. This is the incident review.
Often, when you’re doing something in an interactive shell, you’d like to set an environment variable from a script. But you can’t, because scripts run as their own process, which means that they can’t set environment variables in their parent (the shell). How do we get around that?
I’m trying to figure out how kubectl does its thing. Because it uses HTTPS to talk to the API server, I can’t use
Wireshark to look at the traffic. Here’s how I used socat to snoop on the traffic.
Sometimes, when you run rebar3 shell and you press Ctrl+C twice to quit, your terminal is messed up. This is how I
fixed it in zsh.
In the previous post, I wrote about customizing the Erlang shell
prompt by using the .erlang file. Erlang looks for this file in $HOME (I’m simplifying here); it doesn’t look in the
current working directory. How can we fix that?
Did you know that you can change the Erlang shell prompt?
In this post, I showed how to access the Erlang console via SSH
using kubectl port-forward.
How do I change the title of my terminal window from Erlang?
How do I encrypt (and decrypt) things in Erlang, using an RSA key?
In an earlier post, I showed how to encrypt a JSON Web Token in Erlang. I used plain-ol’ Erlang. Here’s how to use the jose package to do the same thing.
Following on from signing a web token in Erlang, here’s how to encrypt one.
In an earlier post, I showed how to sign a JSON Web Token in Erlang. I used plain-ol’ Erlang. Here’s how to use the jose package to do the same thing.
About 8 years ago, I wrote a post about verifying JSON Web Tokens (JWT) in Erlang. 3 years later, I wrote another post about signing a JWT in bash. This is a post combining the two: here’s how you sign a JWT using Erlang.
How do I sign (and verify) things in Erlang, using an RSA key?
The previous post finished up by writing an RSA public key in “wrapped” (PKCS#8) PEM-format. It occurred to me that I should probably show how to do the same with the private key as well.
I’m currently doing something that needs RSA keys, so here are some Erlang snippets showing how to create one.
I’ve written some code in Elixir Livebook, and I’d like to write some unit tests for it. Here’s how I did it.
I’d like to get some CPU temperature metrics from the nodes in my K3s cluster. That’s the job of Prometheus’
node-exporter. I’m not using Prometheus; I’m using VictoriaMetrics. Here’s how to get it working.
A question on Mastodon asks “What are people using [for cluster management] in 2023?”. I thought I’d address a couple of hidden assumptions in the question and do a quick survey of what’s available.
By default, the improved logger originally introduced in Erlang/OTP-21 doesn’t support per-level colours. This is
something that I miss from lager and from Elixir’s Logger. Here’s a simple
way to implement something like lager.
In the previous posts, we gathered metrics from Cowboy and Hackney. I’d like to publish the metrics to Prometheus.
In the previous post of the series, I added basic metrics reporting to Cowboy, and simply wrote them to the logger. In this post, I’m going to do the same for the HTTP client, Hackney,.
On January 16th, 2023 between approximately 03:15 and 08:00, the router on my home network froze. This is the incident review.
Cowboy is probably the most popular HTTP server for the Erlang and Elixir ecosystem. Here’s how to get metrics from it.
Building and installing Erlang is almost compatible with my use erlang rule for direnv. Here’s how to bodge it.
There’s more comprehensive information in various files in the HOWTO directory, but ain’t nobody got time for that. This works for me.
Some notes about hacking on and contributing to Erlang, because I don’t do it frequently enough to have all of this in muscle memory.
You’re experimenting with Erlang’s built-in SSH daemon, and it fails with “No host key available”. What’s up with that?
The nodes in the cluster stopped talking to each other at some point. I only noticed this afternoon after investigating some other problem.
On January 7th, 2023 between approximately 10:50 and 13:50, there were network connectivity issues on my home network. This is the incident review.
I recently experimented with using Tailscale Subnet Router as a VPN, to allow me to connect to my home network from the pub. It sorta works, but it’s been unreliable recently. I’m going to look at the VPN options that my Synology router gives me.
Given the problems I had when I last upgraded everything on my K3s cluster, I’m going to put a runbook together for doing it “properly”.
Compiling Ruby 2.7.7 fails with the following error about openssl:
It turns out that Traefik has a dashboard. Here’s how to access it via kubectl port-forward.
I want visitors to http://home.k3s.differentpla.net to be redirected to https://home.k3s.differentpla.net. Here’s
how to set that up on K3s, using Traefik middlewares.
For when you want to run both per-suite and per-test setup and cleanup in eunit tests.
This started as a quick experiment to spin up an Erlang cluster on Kubernetes, using TLS distribution, and to validate using cert-manager to issue the pod certificates.
In this post, I showed how to use
an init container to create CertificateRequest objects, which cert-manager signs, returning the certificates. A new
request is created every time a pod starts. This eventually leaves a lot of stale CertificateRequest objects. We
should clean those up.
In a previous post, I used sleep
5s to wait for cert-manager to complete the CertificateRequest. Instead, we should poll the status field.
In an earlier post, I used a
ClusterIssuer that I originally created when first setting up cert-manager. That needs fixing.
Erlang doesn’t like running as pid 0 (usually the init process). I used tini to fix it.
The Erlang daemon that I’ve been using to demonstrate Erlang distribution on K8s is running as the root user. I’m going to fix that.
While scaling up/down the deployment for my Erlang cluster, I regularly refresh the web page that displays cluster members. Occasionally, I get a “502 Bad Gateway” error from Traefik. What’s with that?
In theory, we’ve got TLS working for our Erlang cluster, with mutual authentication. How do we prove that?
In the previous post, I showed how to enable SSH access to the Erlang remote console on a pod. When we left it, it had no authentication. Let’s fix that.
As mentioned earlier, using TLS for Erlang
distribution breaks erlclu remote_console (because it breaks erl_call). At the time, I worked around the problem by
using nodetool. This post shows how to use Erlang’s SSH daemon instead.
When you’re investigating a problem with a deployed application, it’s useful to know precisely which version you’re looking at. Here’s how to automatically set the version number in an Erlang project.
In the previous two posts, we generated signing requests with OpenSSL and submitted them to cert-manager. In this post, we’ll actually use the generated certificates for mutual TLS.
In the previous post we used OpenSSL to create a certificate signing request. In this post, we’ll submit it to cert-manager and get the certificate back.
Erlang TLS distribution is picky about how its certificates are created. OpenSSL is a pain to use. Here’s how to awkwardly wedge the two together. This took me several hours and a lot of my sanity. You’re welcome.
As explained here, I’m going to use an init container to issue the pod certificates.
In the previous post, we got clustering working without TLS. Lifting from the investigation that I wrote up here, I’ll add TLS distribution to my Erlang cluster, but only with server certificates and with no verification (for now).
Based on my investigation with libcluster in Elixir, I’ve decided to use DNS-based discovery for finding the other Erlang nodes in the cluster. To do this, we’ll need a headless service.
I’ve covered this previously; see “Erlang/Elixir Cookies and Kubernetes”. Here’s the quick version.
I noticed that whenever I made any change to the application, it caused the dockerpodman build to re-fetch and
recompile all of the dependencies. On the tiny laptop I was using at the time, this was taking several extra minutes for
every build.
Because this is going to be a cluster of Erlang nodes, there’s (obviously) going to be more than one instance. It makes sense to add some kind of way to have some kind of “whoami” page, so that we can clearly see which node we’re talking to.
For simplicity’s sake, I created a new application with rebar3 new app name=erlclu. I very soon regretted this
decision, because I actually needed a release, so I ran rebar3 new release name=whoops and manually merged the relevant
pieces together.
A few weeks ago, I decided to write a blog post about using mutual TLS to secure Erlang distribution (clustering), with auto-provisioning of certificates when running in Kubernetes. It took a little longer to write up than I expected, and turned into a series of blog posts.
Some notes about using rebar3 with an umbrella project.
On December 14th, 2022 at approximately 10:15, Kubernetes wiped the persistent volumes of a number of applications in the K3s cluster in my homelab. This is the incident review.
While running apt-get upgrade, I was getting a Failed to allocate directory watch: Too many open files error.
You’re using kubectl to do something; you want to do the same using the Kubernetes API (e.g. with curl). How do you
figure out what kubectl is doing?
I’ve been looking for a VPN solution for my home network for a while now, and I recently started using Tailscale.
In a conversation on Twitter about most satisfying code cleanup, I mentioned the time that I completely replaced the agent-inbound HTTP routing at Electric Imp.
I want to set up an Erlang cluster in Kubernetes, using TLS with mutual authentication. This post discusses some of the potential options for doing that. It’s also applicable to general mutual TLS between pods.
In the previous post, I recapped Erlang distribution (clustering). In this post, we’ll secure it by using TLS.
I want to write a post about using mutual TLS to secure Erlang distribution (clustering), with auto-provisioning of certificates when running in Kubernetes. This is not that post. This is a recap of basic Erlang clustering, to refresh my memory and lay some groundwork.
I wrote about Generating a version number from your git repo at the weekend. This post contains some follow-up notes and thoughts.
I’m getting a permission denied error when running brew search:
Problems with zsh on macOS.
I’d like my application to display some kind of version number, ideally derived from the git tag.
Erlang/OTP provides a built-in SSH client and daemon. You can use this to expose the console directly over SSH.
You’ve got an Erlang module with a private (not-exported) function, and you want to add some unit tests for that function? How should you do that?
Usually, when you use localhost, it’ll use the 127.0.0.1 loopback address. Did you know that the entire
127.x.y.z range is reserved for loopback? See RFC 990 and RFC 3330. This can be useful sometimes.
I’ve got an Electric Imp Environment Tail in my office. It monitors the temperature, humidity and pressure. Currently, to display a graph, it’s using flot.js and some shonky Javascript that I wrote. It remembers samples from the last 48 hours.
The documentation for VictoriaMetrics is a bit of a mess, so here’s what worked for me.
Using ArgoCD CLI:
ArgoCD provides a web interface and a command line interface. Let’s install the CLI.
As I add more things to my k3s cluster, I find myself wishing that I had a handy index of their home pages. For example, I’ve got ArgoCD and Gitea installed. I probably want to expose the Longhorn console, and the Kubernetes console. I think Traefik has a console, too. I’ll also be adding Grafana at some point soon.
This is a follow-up to Multiple Laptops, Multiple Monitors; I’ve changed jobs (meaning a new work laptop) and I bought a Surface Dock 2 to go with my Surface Book 3.
The ErlangLS extension for VS Code includes formatting, using rebar3_format. I’d prefer to use erlfmt, so here’s how I set it up.
This afternoon, I fired up my k3s cluster for the first time in a while. When I ran apt update, I got an error message
about a missing Release file.
One of my NUCs keeps hanging, and I suspect temperature. I wanted to use collectd to gather the temperature sensor values, but it was removed from Ubuntu 22.04. Instead, I’m going to see if I can get telegraf to do it instead.
I needed to delete a Glacier vault, which (because it’s not empty) needs the use of an SDK or the CLI. Since I’m using Windows, I figured that I might as well take the opportunity to play with the AWS Tools for PowerShell.
Sometimes you need to install a binary on your empeg. In my case, I wanted strace.
It’s been many years since I last connected my empeg player to a Linux PC, and the muscle memory’s gone. Here’s how to make it work with Ubuntu 20.04.
I’m messing around with Flutter on my nVidia Shield, and I needed to make “tapping” on ListView items work.
Here’s how to debug your apps over the network on an nVidia Shield device.
I noticed that some of my music collection doesn’t have the correct album art. While investigating, I discovered an unrelated problem with the MusicBrainz tags.
I’m attempting to write a Flutter plugin (written in Kotlin). If I open it in VS Code, I get a bunch of “unresolved reference” errors.
Opening a Flutter/Dart/Kotlin project in VS Code on Windows, and a bunch of stuff is broken.
I’m learning Dart and it got me wondering: “Just how many programming languages have you used, Roger?”.
Because I like experimenting with Kubernetes from Elixir Livebook, I made the service account into a cluster admin.
While debugging pod DNS problems, I discovered that CoreDNS allows
customization by importing extra zone files from a config map. I’m going to use that to forward queries for k3s.differentpla.net to my custom CoreDNS instance.
I’ve got an extra instance of CoreDNS running in my cluster, serving
*.k3s.differentpla.net, with LoadBalancer and Ingress names registered in it, and it’s working fine for queries to
the cluster. It’s not working fine for queries inside the cluster. What’s up with that?
I’d like to run Livebook on my cluster. Here’s how I went about doing that.
Up to this point, I’ve been creating and installing certificates manually. Let’s see if cert-manager will make that easier.
There’s a security fix that needs to be applied; there’s an arm64 release candidate. Time to upgrade ArgoCD.
I’ve got Gitea installed on my cluster, but it’s currently accessed via HTTP (i.e. no TLS; it’s not secure).
Because I’m running my k3s cluster on Raspberry Pi 4 nodes, and they’re ARM-64 (aarch64), I keep running into problems where applications are compiled for x86_64 (amd64) and don’t run.
We’re using ArgoCD at work; time to play with it.
My Gitea instance isn’t using TLS, so I’m going to replace the LoadBalancer with an Ingress, which will allow TLS termination.
I’m in the middle of installing ArgoCD (blog post will appear later). Rather than use up another LoadBalancer IP address for it (and mess around with TLS), let’s talk about using an Ingress. It’s entirely possible that I can convert the previously-installed docker registry and Gitea to use one as well.
Sometimes alerts are a bit, well, alert-y. Here are some callouts instead.
I want to play with GitOps on my k3s cluster (specifically ArgoCD). To do that, I’m going to need a local git server. I decided to use Gitea.
If you want to access the Kubernetes API from Elixir, you should probably just use the k8s package, but here’s how to do it without taking that dependency.
I recently left my k3s cluster turned off for a week or so. When I turned it back on, the k3s.differentpla.net DNS wasn’t working. Let’s figure it out and maybe write a runbook for the next time.
If you’ve got separate dev, test and prod environments, it can help to use colour to tell you which one you’re logged
into. Here’s how to colour the iex prompt appropriately.
Distributed Erlang and Elixir applications use a shared secret called a “cookie”. It’s just a string of alphanumeric characters. All nodes in the cluster must use the same cookie. Let’s take a look at what that means in a Kubernetes context.
I’m still on the hunt for a way to connect Erlang nodes in a Kubernetes cluster by using pod names.
I’m looking at setting up an Erlang/Elixir cluster in my Kubernetes cluster, using libcluster, and I’m trying to get my head around some of the implied constraints.
Messing around with the kubernetes API from inside a container.
$ kubectl proxy
Starting to serve on 127.0.0.1:8001
A while ago, I asked why pod names don’t resolve in
DNS, and never really
got a satisfactory answer. One way you can connect to a pod (rather than with a service), is to use the dashed-IP form
of the pod address, e.g. 10-42-2-46.default.pod.cluster.local. Here’s how it works.
If you’re using a NodePort service, and it has multiple replicas, how does it know which replica to use?
I’d like to be able to access my load-balanced services by name
(docker.k3s.differentpla.net, for example) from outside my k3s cluster. I’m
using --addn-hosts on dnsmasq on my router.
This is fragile. Every time I want to add a load-balanced service, I need to edit
the additional hosts file on my router, and I need to restart dnsmasq.
Wherein I finally bring together all we’ve learned so far and stand this thing up properly.
I need to secure my Docker registry. Because OpenSSL sucks, I’m using an Elixir script that uses the ‘x509’ library.
Installation with Helm, per https://longhorn.io/docs/1.2.3/deploy/install/install-with-helm/.
I forgot to disable Klipper, the K3s-provided load balancer.
Installation with Helm, per https://metallb.universe.tf/installation/#installation-with-helm.
To install other things, we’re going to want to use Helm. So let’s install that first.
Having reinstalled all of my nodes with Ubuntu, I need to go back and install k3s. Joy.
My Raspberry Pi 4 cluster is currently 32-bit. It’s got a 32-bit kernel with a 32-bit userland. But I need to run 64-bit software on it. I looked into upgrading it in place, but that’s infeasible. So I need to reinstall it.
Can I upgrade my Raspberry Pi 4-powered k3s cluster to arm64? Without rebuilding everything? tl;dr: no.
While messing around with my k3s cluster, I decided that I needed a DNS server. I tried the Synology DNS Server.
Persistent Volume Claims are namespace-scoped. Persistent Volumes are not:
Pushing a simple node.js-based image to my private docker registry failed.
When you want to push an image to a docker registry, you need to tag it with the registry name. Does the name matter?
Until just now, I didn’t get NodePort services.
In the previous post, we succeeded in giving our docker registry some persistent storage. However, it used (the default) dynamic provisioning, which means we don’t have as much control over where the storage is provisioned.
I wanted to display an alert on one of my pages, like this:
We need to give our Docker registry some persistent storage. Currently, if we restart it, it loses its stored data.
We need to give our Docker registry some persistent storage. Currently, if we restart it, it loses its stored data.
Let’s see if we can push an image to our new Docker Registry.
Yes, it’s a barely-modified Bootstrap 5. I like it, OK?
When I create a new git repository, I prefer for there to be an empty commit at the beginning of history.
The default option for persistent volumes on k3s is local-path,
which provisions (on-demand) the storage on the node’s local disk. This
has the unfortunate side-effect that the container is now tied to that
particular node.
This morning, I grabbed my Raspberry Pi cluster out of the box and fired it up again.
To get the version of the compiler used to compile a particular .beam file:
At home, I’ve got a rather nice setup with two Dell U2520D monitors on my desk, which I’m currently using from my (work) Linux desktop. However, more recently, I’m trying to strike a better work/life balance, so I’ve decided to start making more use of my (personal) Microsoft Surface Book 3 laptop, but I’d like to continue using the two monitors (which are also mine; I never bothered filing the expense claim).
About 2 years ago, I spent some time messing around with k3s on a cluster made from 5x Raspberry Pi 2 Model B nodes.
Erlang/OTP 24.0 added support for ed25519 curves. Here’s some example snippets:
** (Mix) Could not start application example_server: ExampleServer.Application.start(:normal, []) returned an error: shutdown: failed to start child: {:ranch_embedded_sup, :example_server}
** (EXIT) shutdown: failed to start child: :ranch_server_proxy
** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
I recently had to report a bug against ErlangLS (the Erlang Language Server). Here’s how I discovered the version number:
Per Wikipedia:
Ordinarily when writing an SSL/TLS server or client using Erlang/OTP, you’ll use the certfile and keyfile options, as follows:
I’ve just spent about a day poking around in the guts of Erlang/OTP and Ranch, and I thought I’d write some of it down.
Signed 32-bit time_t, which counts seconds since 1970 runs out after 2^31 seconds; i.e. in 2038.
After upgrading to Ubuntu 20.04 at the weekend (this is not strictly relevant), one of my builds started failing with the following error:
At Electric Imp (now part of Twilio), my team uses Erlang’s Common Test for driving our system tests. These are (almost-)end-to-end tests that exercise (almost) the whole platform.
I’ve got my printer connected to a Raspberry Pi print server, and printer discovery only occasionally works from ChromeOS. Here’s an aide-memoire for configuring it.
I’ve got a bunch of Erlang nodes running in Docker containers, and I’d like to connect a remote shell, running on the host, to one of them.
By now you should be using Python 3.x. You can integrate it with direnv (2.21 or newer)
by putting the following in your .envrc
Note: This is basically the same as the node.js server.
Quick reference for installing Erlang and Elixir on a Raspberry Pi, using the Erlang Solutions packages.
Now that I’ve successfully run nginx on my cluster, I’m going to do the same with a simple node.js server.
Start nginx, with 3 replicas:
It’s at this point that I diverge from Scott’s blog post; he installs full-fat Kubernetes. I’m going to use k3s.
I downloaded Raspbian Buster Lite from the official site and wrote it to the SD cards (using dd).
I had a bunch of hardware lying around from an earlier abandoned side project:
I’m reading “Practical Haskell”, which requires a different version of Haskell than Ubuntu installs so I figured I’d add another entry to my direnv series.
Recently I noticed that I’d made some commits to the git repository at work with my personal email address.
Visual Studio Code is unable to watch for file changes in this large workspace
I installed the following extensions:
I have more than one Google account, and this confuses Google apps.
use is a way that you can inject code into your module. Elixir uses this for boilerplate in GenServer, and Phoenix uses it for metaprogramming.
When an error occurs in the Electric Imp backend, it’s logged and collated, and we regularly review error reports to see if there’s anything that needs looking at.
This morning, something electrical went wrong in my “network cabinet” (hall cupboard), which knocked me offline for about an hour. Here’s the post-mortem.
To integrate direnv with ruby-install – installation instructions here – add the following to ~/.direnvrc:
To integrate direnv with python-build – installation instructions here – add the following to ~/.direnvrc:
Install pyenv. We’re not going to use it as-is, but it includes the python-build script that we will use:
To integrate direnv with nvm, add the following to ~/.direnvrc:
At Electric Imp, we manage our Elixir versions with kiex.
\curl -sSL https://raw.githubusercontent.com/taylor/kiex/master/install | bash -s
To integrate direnv with kiex, add the following to ~/.direnvrc:
Use this helper function:
At Electric Imp, on developer PCs, we manage our Erlang versions with kerl.
To integrate direnv with kerl, add the following to ~/.direnvrc:
If you want to use kerl to build your Erlang installation, you’re going to
need some packages installed first.
From direnv.net:
At Electric Imp, we use a variety of different languages, and each project might be using a different mix of versions.
If kerl list installations is displaying Erlang installations that you
deleted ages ago, and you’ve got all of your installations in
~/.kerl/erlangs/, you can rebuild the list by running the following command:
I last posted on this topic about 4 years ago. It’s time for an update.
I found myself having to process a large JSON file using jq --slurp, and it was using up a lot of memory, so I thought it was time to learn about jq’s reduce function.
When you use gen_server:start/3,4 or gen_server:start_link/3,4, the call
blocks until the other process has finished running init/1.
Installing the script to check for The ROBOT Attack. On Linux Mint 18.3. Yak shaving.
This is an internal email I wrote almost exactly 2 years ago. I had to refer to it today, and I figured that it bears sharing publicly.
Previously, I posted about a bug we’d fallen foul of in libgnutls28 on Ubuntu 14.04. There’s a patch.
I wanted to get Erlang running on my Synology NAS, for various reasons, and was struggling with the cross compiler.
I don’t recommend enabling remote SSH access to your router, but if you need to, here’s how to do it via the web interface (LuCI):
I’m currently playing with OpenID Connect (OAuth 2.0 for Login), to allow people to log into a web site using their Google account. The web site is built using Erlang.
When setting up a Mailgun domain, you need to configure some DNS entries, shown as follows in the Mailgun console:
On my desk I have a dashboard (an “information radiator”) that displays various useful things, such as the build status of various Jenkins jobs, clocks showing important timezones (Electric Imp is a distributed company), and so on.
Attempting to sudo bash, I was confronted with:
We use lager for our logging at Electric Imp. This morning I had cause to tweak the configuration at runtime on one of our staging boxes, but first I needed to figure out which handlers were already installed.
In Erlang, in a gen_server, when does terminate get called? Also, some
messing around with dbg for tracing.
I find Erlang’s gen_event behaviour to be fairly tricky to understand,
despite the copious documentation on the subject:
This question just came up on Twitter:
It turns out that you can’t load Erlang NIF libraries from the shell.
We’ve found a couple of problems with Erlang R16 that mean that one of our projects requires Erlang 17. Conversely, we’ve found a couple of problems with Erlang 17 that mean that one of our other projects should stick with Erlang R16.
kerl allows you to easily build and install multiple Erlang/OTP releases.
It’s kinda like nvm or rvm, but for Erlang. It doesn’t do everything, and
that’s what direnv is for. direnv allows you to run commands upon entering
or leaving a particular directory.
While using erlang.mk and relx to build a newly-created Erlang application, I got the following error:
Originally posted to the Electric Imp blog. Preserved here.
You’re using lager for logging in your Erlang program, and you discover that
your configuration isn’t logging anything more verbose than info-level
messages. How do you bump that up to debug?
If you’re connected to an Erlang node via a remote shell, and you don’t have access to stdout on the original node, you’ll need to redirect the trace output to your current shell.
It’s pretty simple to create a ZIP file on disk.
I’m writing a simple web application in Erlang that uses
epgsql to talk to a PostgreSQL database. I keep
getting invalid_password, despite the fact that the user doesn’t have a
password (and I’m passing “”). I can also log in by using psql -U foo.
I’ve been using Syntastic for a while now. It’s awesome.
This, for no particular reason, is the list of vim plugins that I’m currently using:
If you don’t know what electric imp is, what an April Development Board is, and you don’t have a Dymo LetraTag LT-100H label printer, then this isn’t going to be of much use to you.
I’ve got my TERM environment variable set to xterm-256color
(even though I’m using gnome-terminal, but that’s a different story).
When I ssh into my OpenWRT router, I get presented with the error message:
This is mostly a reminder to myself.
Some odds and ends.
handle_event and handle_sync_eventIn PowerShell, when I attempted to enumerate SSL bindings, I was getting the error message:
Just some random thing I learnt today while playing with Erlang:
My son has quite a lot of Lego, but he stopped playing with it. It turns out that he gets frustrated when he can’t find a particular piece. So we started exploring storage options.
Cowboy protects itself against slow requests by doing the following:
I was attempting to use SSH to set up a tunnel (to our build server) this morning, so I typed the following:
I’ve been working for electric imp for about three months now, and I’ve been planning to articulate why I made the change, or more specifically, how it meant getting way out of my technical and working comfort zone.
node.js modules are singletons. If you require('foo') from one module, and
then require('foo') from a different module, you’ll get the same instance.
The documentation is for Java, and mentions the PKCS10CertificationRequest class.
I was attempting to convert some Java code that uses Bouncy Castle into the
equivalent PowerShell. The Java code was using the PKCS10CertificationRequest class,
and I needed to see if there was an equivalent in the .NET libraries:
Apropos of a discussion on Facebook earlier today:
While attempting to debug a crash dump from a .NET 4.0 RTM process on my (.NET 4.5), I got the dreaded
Failed to load data access DLL, 0x80004005 error message.
tl;dr? Skip to the end…
In full, the error looks like this:
0:000> .loadby sos clr
0:000> !vmstat
Failed to load data access DLL, 0x80004005
Verify that 1) you have a recent build of the debugger (6.2.14 or newer)
2) the file mscordacwks.dll that matches your version of clr.dll is
in the version directory or on the symbol path
3) or, if you are debugging a dump file, verify that the file
mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.
4) you are debugging on supported cross platform architecture as
the dump file. For example, an ARM dump file must be debugged
on an X86 or an ARM machine; an AMD64 dump file must be
debugged on an AMD64 machine.
You can also run the debugger command .cordll to control the debugger's
load of mscordacwks.dll. .cordll -ve -u -l will do a verbose reload.
If that succeeds, the SOS command should work on retry.
If you are debugging a minidump, you need to make sure that your executable
path is pointing to clr.dll as well.
There are lots of words here, but they’re not the easiest to understand, particularly when you’re in a hurry (like, say, you’re attempting to debug an issue on a production server).
I’m assuming that you’ve got a recent version of WinDbg. I was running 6.12, which should be OK.
If you’re lucky, however, (and you’ll need to have !sym noisy turned on to see this), you’ll get:
DBGHELP: clr - public symbols
c:\websymbols\clr.pdb\0A821B8A573E42899202851DF6A539F12\clr.pdb
SYMSRV: mscordacwks_AMD64_AMD64_4.0.30319.01.dll from http://msdl.microsoft.com/download/symbols: 502605 bytes - copied
DBGHELP: c:\websymbols\mscordacwks_AMD64_AMD64_4.0.30319.01.dll\4BA21EEB965000\mscordacwks_AMD64_AMD64_4.0.30319.01.dll - OK
This means that the Microsoft Symbol Server had the relevant mscordacwks.dll
file. If not, you’ll have to copy it from the machine from where the crash dump
originates, and put it somewhere useful. Unfortunately, the message doesn’t
explain what, exactly, it means by “on your symbol path”.
For me, at least, it seems that putting it in (e.g.) C:\WebSymbols doesn’t
work – you need to put it in the correct named location. I can’t verify this
right now, since it downloaded it successfully from Microsoft Symbol Server.
More importantly for my problem, however, is the easily-overlooked message at the end:
If you are debugging a minidump, you need to make sure that your executable
path is pointing to clr.dll as well.
Given that WinDbg knows that I’m debugging a minidump, it could have made this more prominent. What it’s telling us is this:
.exepath+ C:\whereverNote, that you must use the version from the original machine. If you
attempt to use your clr.dll, you’ll get the message:
The version of SOS does not match the version of CLR you are debugging. Please
load the matching version of SOS for the version of CLR you are debugging.
CLR Version: 4.0.30319.1
SOS Version: 4.0.30319.17929
CLRDLL: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscordacwks.dll:4.0.30319.17929 f:8
doesn't match desired version 4.0.30319.01 f:8
clr.dll, sos.dll and mscordacwks.dll from the machine where the
crash is occurring.C:\Temp..exepath+ C:\Temp.load C:\Temp\sos!vmstat, etc., to your heart’s content.So, why did I use IIS Express to implement my vs-welcome-page extension? I’m using Nancy, so why didn’t I just use the self-hosting support and host it inside Visual Studio instead?
Next problem: packaging the vs-welcome-page extension.
A brief aside: I’ve got to the point where I’m about to fire up IIS Express from Visual Studio. In order to be a good citizen, our extension should probably shut it down again later.
The next major piece is the Visual Studio extension, so let’s get on with that. Assuming that you’ve installed the Visual Studio 2012 SDK, it’s as simple as creating a new “Visual Studio Package” (it’s under “Visual C#” / “Extensibility”):
There are several options for rendering Markdown to HTML in .NET. If you bring
up the NuGet Package Manager dialog and search for “markdown”, you get three
pages of results, including MarkdownSharp, which is the renderer used by
Stack Overflow, so it must be good.
Using the web browser that’s built into Visual Studio is dead simple:
An aside: We need to be able to support multiple instances of our small web application (because you might have more than one instance of Visual Studio open). If we host our application in IIS Express (and remember that we’re doing this to avoid messing around with URL ACLs), we either need to duplicate the whole thing for each site, or figure out a way to provide it with parameters.
When you create a web application in Visual Studio, you can configure it (on the properties page) to run in IIS Express.
I wanted to write a Visual Studio extension (for VS2012) that, when you open a solution, renders and displays Markdown documentation in Visual Studio.
We had an issue at work last week where a web application was taking a little bit too long to start up, which was causing intermittent failures. We’d like the application to be always-running, so that this doesn’t happen.
How do we find out if there’s a worker process for an application pool? Note that there can be more than one worker process for each application pool. Note also that a worker process can host multiple web applications.
Having recently moved my home server (Windows 2012) from the hall cupboard to the corner of my study, I decided to connect it to my printer. This means that I can print from my laptop over wireless without needing to turn on the desktop PC that the printer was originally connected to.
As I run across them, I’m going to update this page with various causes of the WCF MessageSecurityException.
Again, this is more-or-less a direct port from the C# post.
Note: This is a bit long, because I want to take a moment to show some of the problems you might have using PowerShell to call .NET code that’s written in a certain style.
PowerShell has a really cool feature where it’ll coerce a string into an enum value. I’ll demonstrate by exporting a certificate from the Windows certificate store:
This is all just .NET reflection code, converted from C# to PowerShell, but it can be useful…
I recently wrote a series of blog posts on how to use the cryptography libraries from the Legion of the BouncyCastle in C#.
I just bought myself a shiny new Acer Aspire S7-391, with a view towards nuking Windows 8 and to install Linux Mint on it. Stupidly, I didn’t consider that Windows 8 logo certification now requires a UEFI BIOS and Secure Boot.
Over the last few posts, we’ve seen how to create a self-signed server certificate in C#, by using the Bouncy Castle library. How do we create a CA certificate, and how do we issue certificates from that authority?
When you connect to a server using HTTPS, the server provides a certificate that identifies it. Your browser will then typically check that the name in the server certificate matches the address that you’ve gone to. In this way, you can be sure that you’re connecting to the correct server.
If you look at a certificate, you will see something like one of the following:
Over the last couple of days, we’ve seen how to use Bouncy Castle to generate certificates from C#. However, if you compare the certificates we’ve been generating with those generated by (e.g.) makecert, you’ll see that we’re missing a few things.
In the last installment, we created a Bouncy Castle certificate. How do we now create a .NET X509Certificate2 object?
Occasionally, you might find that you need to create a self-signed server certificate. If you’re on Windows, your options are basically:
I’ve just spent the bulk of the morning attempting to understand a bug by reading through some fairly large log files. I’ve come to the realisation that:
Most log files are text. This means that, if you’re tracking down a problem, you’ll probably want to search them, and you may want to edit them.
In particular, you may want to do a search+replace to pull out a bunch of lines to do with a particular event or client. For this reason, don’t spread an event across multiple lines. You might think that it makes the file easier for a human to read (and it does), but it makes it impossible to (for example) delete all of the events that mention client 321 (because you’re interested in client 123), because you’ll end up with hanging lines, as in the following example:
Client #123: Getting orders...
Order #1: Foo
Order #2: Bar
Client #123: Returned 2 orders.
Client #321: Getting orders...
Order #3: Baz
Client #321: Returned 1 order.
If you search-and-replace to delete “Client #321”, you’ll end up with the following:
Client #123: Getting orders...
Order #1: Foo
Order #2: Bar
Client #123: Returned 2 orders.
Order #3: Baz
If you tag-and-copy-to-new-file for “Client #123”, you’ll end up with the following:
Client #123: Getting orders...
Client #123: Returned 2 orders.
Either way, you’ve lost information, and confused yourself. This makes it harder to understand what’s going on.
So, either include the context on every line:
Client #321: Getting orders...
Client #321: Order #3: Baz
Client #321: Returned 1 order.
Or, don’t spread your output over multiple lines.
Yeah, your log files are human readable. That’s great, provided you can find a human prepared to read through a couple of million lines of log file.
Why don’t you make them machine-readable? Then you’d allow the file to be parsed and processed by a vast number of tools that already understand CSV (or JSON).
Whatever you do, don’t use XML. If you do, you’ve got two choices:
</root>
before every write. Say goodbye to your performance.Also, it’s massively verbose. Just use CSV or JSON.
Obviously, this isn’t going to work for your web-scale application, because you’ve got millions of clients visiting every second. However, if you’ve only got a couple of dozen clients, consider using one log file per client.
System.InvalidOperationException: Binding validation failed because the WSHttpBinding does not support reliable sessions over transport security (HTTPS). The channel factory or service host could not be opened. Use message security for secure reliable messaging over HTTP.
…reliable sessions and HTTPS don’t mix.
What’s the PROCESSOR_ARCHITEW6432 environment variable for?
Last night, I kicked off an upgrade from Ubuntu 10.10 (Lucid Lynx) to 12.04 (Precise Pangolin). I am distinctly unimpressed by the upgrade process.
Yesterday, I said that I’d be migrating my website to a virtual machine; originally, I’d planned to mess around with Linux Live CDs.
As part of implementing my new blog engine, I want to decommission the box that’s hosting my existing blog. It’s taking up space in the cupboard, and I’ve got a better use in mind for the hardware.
It turns out that you cannot manage a Windows 2008 R2 Hyper-V installation from a Windows 8 desktop (even with the Remote Server Administration Tools for Windows 8 installed) or from Windows 2012 Server.
While looking through some MSBuild scripts recently at work, I came across some stuff to do with Task output parameters that I thought might be worth blogging about. First, however, I need to talk about Task input parameters.
enum types over constants or propertiesAttempting to use Add-Type to load a .NET assembly in PowerShell this afternoon, I got the following error:
Attempting to use Add-Type to load a .NET assembly in PowerShell this afternoon, I got the following error:
$certs = Get-ChildItem cert:\LocalMachine\My | where { $_.Subject –like 'CN=Victim*' }
foreach ($cert in $certs) {
$store = Get-Item $cert.PSParentPath
$store.Open('ReadWrite')
$store.Remove($cert)
$store.Close()
}
Sometimes you have a web service or web application, and, for one request, you need to transfer an unknown (and potentially unbounded) amount of data in the response.
```powershell
$x = ‘A’ $sb = { Write-Host $x }
I recently started having odd problems with SQL Server Management Studio on my work PC.
Some programs write their output in different colours. If they’re badly-behaved and don’t reset the colours when they’ve finished, your PowerShell console might be left with the wrong background or foreground colours.
I thought I might have a quick play with node.js on Windows. There’s a bunch of conflicting information out there about how to install it. Here’s what works for me.
New theme: Acquia Slate. It’s maybe a little dark for my taste, but I’m going to leave it for a couple of weeks and see if it grows on me.
I’ve been using PowerShell as a better Command Prompt for a while now. Increasingly, however, it’s one of the first tools I reach for when I come across a problem, rather than being an afterthought.
I’ve not posted this until today, because my blog’s been poorly, but the BT engineer came round last week and upgraded me to FTTC.
Talking to @ptrelford at the TDD & Refactoring workshop run by Jason Gorman, Phil mentioned that he’d implemented a really simple IoC container in F#. “But there’s a really simple one already built into .NET” says I…
Somehow – I’m not sure how, and I’m not exactly sure when – my website (powered by drupal) lost all of its images. They were still there in the filesystem, and the rows were still in the node table, but they’d vanished from the actual pages.
If you’ve got an enumerable, and you want to pass it to SqlBulkCopy, you can turn it into an IDataReader. Something like the following might be useful:
I’m using SqlBulkCopy to insert a large number of rows into a database table. I used Observable.FromEvent to hook the SqlRowsCopied event. Rx provides the .TimeStamp() method, but I wanted to know the elapsed time, so I did the following:
I’m spiking a WCF service hosted in IIS this weekend. It uses SQL Server, and I was getting the following error in WCF Tracing:
Described here.
So, I thought I might learn F# by doing some Code Kata. I’ll start by doing the exercises from codekata.org. Unfortunately, they don’t have any F# templates, which means I’ll have to improvise at points.
It’s actually quite simple:
This is related to “Can you make a build in one step?” on The Joel Test. But it’s not the same.
Since your code signing certificate probably lasts for two years, you’ll have forgotten what to do with it when you receive your new one.
One of the cool things you can do with the Microsoft Debugging Tools is set up a source server. This works by embedding source control details in your PDB files. Once you’ve published these to a symbol server, and set that up properly, you should be able to load a minidump file in Visual Studio (or WinDbg) and be taken to the correct line of the correct revision of the source file where the crash happened.
One of the many things that my former company, 1E, does well is its build and release process. That got me thinking: what makes a good build process?
I’m messing around with writing yet another a blog engine, as a way to learn ASP.NET MVC. One of the things that I’d like to do is have it support uploading from Windows Live Writer. This means that it needs to support XML-RPC and Really Simple Discovery (RSD). More on the XML-RPC stuff later.
When you use P4INDEX (or SSINDEX -System=P4), you might see “zero source files found”. Check that your workspace doesn’t have forward slashes in the name (e.g. D:/Source/depot/).
The key used for signing your releases should be kept secure. It should be impossible for a rogue developer or 3rd party to use this key unofficially.
This is just a minor rant:
These are the extensions that I have to install whenever I set up a new copy of Google Chrome.
NBehave depends on NUnit 2.5.2. To get it to work with NUnit 2.5.5, you can create an NBehave-Console.exe.config file containing the following:
In attempting to see if I could get my NBehave tests working inside Visual Studio 2010, I stumbled across Gallio, which, as far as I can tell, claims support for ReSharper and NBehave.
At the weekend, I picked up a shiny new HTC Desire, which has first-class support for GMail, including Google Apps. It also has support for Exchange ActiveSync.
We’ve just started using NBehave, so this is a random selection of notes:
The Visual Studio 2010 splash screen is stored in C:\ProgramData\Microsoft\VisualStudio\10.0\vs000223.dat. It’s a hidden file. It’s a PNG file.
The Visual Studio 2010 splash screen
is stored in C:\ProgramData\Microsoft\VisualStudio\10.0\vs000223.dat. It’s a hidden file in PNG format.
Because whenever I’ve been away from SQL Server for more than a couple of weeks, I forget.
Instead of this:
Publish to Digital Photo Frame is a plug-in for Windows Live Photo Gallery that makes it easy to upload your photos to a digital photo frame.
My desktop PC is Windows 2008, with Hyper-V disabled. I’ve got a Windows 2008 R2 box to act as a Hyper-V server. I created an administrator user account with my name on the R2 box, and used HVRemote to allow remote access.
If you’ve installed squid on a box, you’ll probably want to use it.
At work, we do a lot of our testing and debugging with virtual machines (using Microsoft Virtual Server or Hyper-V, mostly). The virtual guests are generally not allowed any access to our live network, which makes accessing the Internet difficult. To get around this, I usually set up Squid on the host machine. This allows the guests to access (e.g.) Windows Update and Microsoft Symbol Server, which makes things a bit easier.
Marc Gravell wrote about using Expression<T> as a compiler. It was a bit of an eye-opener.
Some random notes:
The hosting outfit that I’ve been using for differentpla.net is cheap for a reason: they’re not very good.
If you’re using the Microsoft Symbol Server to pull down symbols for Windows, or if you’re using SYMSTORE.EXE for your own stuff, you’ll see that EXE (and other binary files) are stored in a path similar to C:\WebSymbols\user32.dll\4226015990000\user32.dll.
I’m making some improvements to our build scripts at work, and I ran into a problem where Source Server indexing wasn’t working.
If you want to use Castle Windsor’s MicroKernel (or another IoC container) for ASP.NET MVC controllers, it’s fairly easy. The ASP.NET MVC team allowed you to replace the factory used to create controller objects.
For some reason, LINQ’s ForEach extension method doesn’t work on IEnumerable<T>; it only works on IList<T>. Easy fix:
I was doing some ASP.NET MVC this weekend, and I couldn’t get the project to start: all I got was “Internet Explorer cannot display the webpage”, when using Visual Studio’s built-in development web server (Cassini).
…because then you can’t (easily) use Console.Write, because “Console” clashes.
Ayende shows how to use SQLite with an in-memory database for unit testing your NHibernate code (here). This is a great idea: your unit tests will run more quickly, and you don’t have to worry about tearing the database down when you’ve finished. On the other hand, it’s not an exact match for Microsoft SQL Server (which is probably what your production website will run against).
At the end of February, differentpla.net suffered filesystem corruption. I had to revert pretty much everything to a previous backup, which meant losing about a week’s worth of stuff.
When I’m supreme ruler of the universe: shops will be required to place the “house” number somewhere on the front of the shop. It should be easily visible.
Nothing to report. It all works fine. Installation took an hour. The only hiccup was that I had to download the Marvell Yukon drivers (I used the Vista x86 ones) from Marvell’s website, but the WiFi works, so that was easy. I also slapped an SDHC card in there for some ReadyBoost goodness.
This seems to be causing a few problems, so I’ll quickly walk through it. I’ll try to throw some screenshots in later.
On Windows, 64-bit processes cannot load 32-bit DLLs, and 32-bit processes cannot load 64-bit DLLs. How does REGSVR32.EXE manage to successfully register both 32-bit and 64-bit COM DLLs?
SetWaitableTimer doesn’t accept NULL for the pDueTime parameter.
SVCHOST services are configured by having ImagePath set to "%windir%\system32\svchost.exe -k _name-of-service_",
and a Parameters key containing ServiceDll (REG_EXPAND_SZ), which names a DLL with a ServiceMain entry point.
ServiceMain has argc and argv.
RegQueryValueEx, when passed lpData = NULL, will set *lpcbData to the length required in bytes, even if it’s
already set to something. If lpData != NULL, and *lpcbData is too short, RegQueryValueEx will return
ERROR_MORE_DATA.
RegSetValueEx, when passed REG_SZ, needs the length to be in bytes, so don’t just use _tcslen like this:
Because Visual Studio doesn’t look for references in the GAC:
Because I keep forgetting how to.
In response to this thread on the empeg BBS, I hacked together a couple of small tools for the empeg car player.
I’m at DevWeek this week, and I went (among other things) to a couple of WCF presentations by Aaron Skonnard.
Essentially, you write a new task like this:
I just started using TeamCity Professional Edition for some personal projects (and we’re evaluating the Enterprise edition for use at work).
When I’m supreme ruler of the universe: If you’ve taken more than 5 days vacation, employers will be required to let you work only a half-day on your first day back. It could be the morning, or the afternoon, but you’ll be required to work only 4 hours.
To the tune of the twelve days of Christmas…
The last couple of entries have shown the hex dumps of the individual packets. We’ll step back to a slightly higher-level view now.
The next packet that the empeg sends is the actual response to the OP_STATFID request. It looks like this:
We’ve seen the format of the packet that emplode sends to find out the length of a file on the empeg (OP_STATFID). The empeg seems to always send a progress packet in response. The packet looks like this:
The format of the packets in the empeg’s TCP/IP protocol owe a lot to the fact that the empeg mk1 communicated over a serial link. This means that there’s a certain amount of cruft still in the protocol.
You should hide your notification icon unless you’ve got something important to tell the user. More and more applications insist on putting mysterious icons in the notification area, and most users have no idea what they’re for.
Like it says: NIM_SETVERSION always returns FALSE unless you’ve used NIM_ADD. So send NIM_ADD first.
A short while ago, I asked whether it was possible to debug system processes without administrative privileges. I ran into a brick wall and couldn’t figure it out.
There’s probably a better way to do this, but here’s what I came up with in a couple of minutes…
I’ve been running with a normal, non-administrator (LUA) account for the past 2 years or so, without any real problems. Most of the time, the problem is simply that you can’t install something, or register something, or you need to install a service.
Unless specified otherwise, all source code published on the differentpla.net website is available under the following license terms: Do what you want with it, but don’t claim you wrote it, and it’s not my fault if it doesn’t work.
I recently had a problem getting STARTTLS working on exim4 on Debian 4.0 (Etch). It kept reporting:
``` class FibonacciSequence : IEnumerable { public IEnumerator GetEnumerator() { yield return (long)0; yield return (long)1;
If you want to read a file and bypass security (e.g., if you’re writing a backup program), it’s not enough to simply enable the SE_BACKUP_NAME privilege, you have to also pass FILE_FLAG_BACKUP_SEMANTICS to CreateFile.
Let’s assume that you’ve got a table:
Here’s how to find the time 30 minutes ago:
Apparently, you can debug JScript in Visual Studio.
It’s possible, through the magic of COM interop, to call C# code from JScript or VBScript. Here’s an example of how to do it from JScript.
function forEach(enumerable, delegate)
{
for (var enumerator = new Enumerator(enumerable); !enumerator.atEnd(); enumerator.moveNext())
{
delegate(enumerator.item());
}
}
As far as I can tell, you’ll have to go “old school”, and do the equivalent of adding lots of printf statements to
your code. This should help:
Some gadgets allow the user to configure extra settings.
The Sidebar Gadgets SDK documentation recommends 12 pixel Segoe UI
When you click on the “Show details” expander in the Windows Vista gadget gallery, Windows displays various information about the selected gadget (“Calendar 1.0.0.0” etc., in this example).
When your gadget is docked, it displays a background image, which is allowed to be alpha-transparent.
When you add a new gadget to the sidebar (by right-clicking and selecting “Add Gadgets…”), Windows Vista displays an icon for it.
At work, I do all of my development on a Windows 2003 box. I generally log in as a normal user, and then use Remote Desktop to concurrently log in as Administrator. This means that I’m relatively safe from malware when doing day-to-day stuff, but that I can still get hold of Administrator privileges when I need them.
I started getting this error on one of my Windows 2003 boxes yesterday:
Shame that the DrawEdge() documentation doesn’t include examples. Fixed.
I’ve got a dialog box with a SysLink (WC_SYSLINK) control on it. When the user clicks on the link, it should open our website in the user’s default browser. In my test app, it works. In the production app, it doesn’t.
…because every time I search for a given HRESULT, all I get is yet another frigging VB website regurgitating the contents of WinError.h
On Windows Vista, applications running without admin privileges can write to HKEY_LOCAL_MACHINE, and they’ll be transparently redirected to HKEY_CURRENT_USER\Software\Classes\VirtualStore. Similar redirection applies to C:\Program Files.
Someone appears to have spent some time searching my site for the term “Roman Numeral letter H”. This is pointless, for two reasons:
/manifestdependency linker switch. As with several other linker switches, this can be done using #pragma comment(linker, ...)./manifestdependency switch into .OBJ files that it creates. It marks them as dependent on the Microsoft.VC80.CRT DLL, version 8.0.50727.762. It does this in the VC\include\crtdefs.h file.C:\Windows\WinSxS folder. They both have extremely long names. For example, the latest one lives in a folder called x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.762_x-ww_6b128700.In the C:\Windows\WinSxS\Policies\x86_policy.8.0.Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_x-ww_77c24773 directory, there’s a couple of policy files. The first one says that any CRT versions between 8.0.41204.256 and 8.0.50608.0 should be redirected to version 8.0.50727.42. Presumably this covers a range containing all of the CTPs and betas up to the RTM version.0x80040200 is SELFREG_E_TYPELIB. This result is returned from MFC ActiveX (.OCX) controls when AfxOleRegisterTypeLib fails.
I’ve recently been struggling to reliably send email via my qmail server. I’ve got STARTTLS and SMTP AUTH enabled, in order to allow relaying for authenticated users. However, it’s been really slow, and Thunderbird quite often times out and gives up on the connection.
Because I might want to get on with something else while your program loads, and if your splash screen is in the way, that’s going to make it difficult.
Unable to open Web Project 'foo'. The file path 'C:\wherever\foo' does not correspond to the URL 'http://localhost/foo'. The two need to map to the same server location.
When moving differentpla.net from one server to another, I decided to move the content (currently served using drupal)
from the root to /content. I also decided to use more descriptive URLs.
I’m doing a bunch of SQL hacking at the moment, and I’ve decided to use SQL Query Analyzer as a good template for how not to write a text editor.
When you import a project from Visual C++ 6 into Visual Studio 2003 (i.e. Visual C++ 7.1), it doesn’t fix up the output directories to use the correct definitions.
When compiling a C++ program with Visual C++ 2003, you might get the error:
Sometimes, when attempting to debug an ASP.NET Web Service, you’ll get the following error:
I recently had my machine at work rebuilt with Windows 2003, and had a spot of difficulty getting Visual Studio to debug my ASP.NET applications. Here are the various error messages I encountered, and their fixes.
I was just hacking something together in Visual Web Developer Express, and was attempting to add a database to my App_Data directory when I ran into problems.
If you try using IDictionary with the PropertyGrid control, the results aren’t spectacular:
In this installment (see here for the previous installment), we’ll be fixing a few things and making the whole thing prettier.
Oddly, the Windows Forms libraries don’t provide any support for writing wizards. Here’s one way to do it.
If you’ve ever gone looking for the music and playlists on your empeg, you’ll find them in the /empeg/fids0 and /empeg/fids1 directories.
Several popular applications implement their options dialog as a collection of pages. Here’s one way to do this in your application.
A couple of weeks ago, I installed a trial copy of the Dundas .NET charting controls to play with in ASP.NET. It’s actually pretty cool, but I was attempting to show it to someone on my laptop when it stopped working.
In Dino Esposito’s “Cutting Edge” column in MSDN Magazine, he talks about the various readers (BinaryReader, XmlReader, etc.) available in .NET.
Imagine this code:
While writing some ASP.NET, I got this error. I’ve just spent ten minutes figuring it out, so I thought I’d share.
It turns out that Visual Studio.NET Professional can’t be used to design databases with SQL Server. It can be used with MSDE.
Obviously, your code doesn’t just link with your libraries. It also has to link with some of the system libraries. Jam manages this by using the LINKLIBS variable. The simplest way to make this work is something like the following:
In an earlier article, Running Long-lived Tasks in a Background Thread, I talked about some of the problems associated with reporting progress from a background thread in a Win32 application.
If you’ve got a LISTBOX (as opposed to a ListView) control in a dialog, and you’re using LBS_OWNERDRAWFIXED, you might be surprised to find that your MeasureItem function is never called.
A common UNIX shell idiom is to write things like cwd=$(pwd) to save
the current directory so that in case be used later when other things might
have changed it. For a long time I didn’t think this was possible in Windows
batch files. It turns out that it is, but it isn’t pretty and it does
generalise to getting the output of any command.
In my previous article, I explained how to use LPSTR_TEXTCALLBACK and LVN_GETDISPINFO.
If you specify LPSTR_TEXTCALLBACK when inserting an item into a ListView control, it no longer supports sorting; you can’t click on the column heading to sort the list control.
Over on the empeg BBS, tfabris said:
In an earlier article, I showed how to draw a drop shadow with the help of the ImageList control.
The documentation for ImageList_DrawIndirect seems to imply that it can draw an item from an ImageList control with a drop shadow. Unfortunately, it doesn’t seem to be supported.
Inspired by this post on Raymond Chen’s blog, I thought I might explain how to change the logon pictures.
In the Grabbing a snapshot of the screen article, I said: “I’m planning on altering the snapshot before displaying it.”. This article shows how to alter a bitmap by directly accessing the raw bits.
Some (simple) code that shows how to grab a snapshot of the screen and display it in a window.
I got this error message today:
Some twisted code showing how to combine MFC’s CTreeCtrl, CCmdTarget and TrackPopupMenu.
If you’re allowing the user to “rubber-band” a selection in your application, you might choose to use the DrawDragRect function. Here’s how.
The sequel to Partitioning an empeg’s Hard Disk Manually.
If you’re attempting to upgrade the hard disk in your empeg, and the disk builder is not working for you (this is often the case with disks larger than 60Gb), then you might want to build your new disk manually.
To make it easier to manage the virtual domains on my email box, I’ve decided to install qmailAdmin.
By default, Outlook stores a copy of your outgoing email in the “Sent Items” folder in your local “Personal Folders” .pst file.
If you install this patch (mentioned here), then you’ll get SMTP-over-TLS in qmail.
Whoops. Back when I was installing ClamAV, I
started clamd running so that I could test the virus checker. Yesterday, I rebooted the PC.
Outlook Express likes to store a copy of your outgoing email in a folder called “Sent Items”. Thunderbird likes to store a copy of your outgoing email in a folder called “Sent”. If you use both to access the same email account using IMAP, you’ll end up with both folders being created, and your outgoing email will end up in either, depending on which client you used at the time.
BincIMAP supports IMAPS for communication. It can either do this if you compile in SSL support, or if you use an SSL tunnel, such as stunnel or ucspi-ssl.
On my list of “things to do in my copious free time” is: “write a replacement Rio Receiver server in C#”. So I’ve started putting together snippets of code that might turn out to be useful as I start learning C#. This one is a recursive file finder. It uses delegates.
I’ve been playing around with learning C#. Today, I thought I’d write a C# program to find empegs on the local network.
At the end of Installing SquirrelMail, I’d finished installing webmail on my test box. Currently, this uses HTTP. This is not really secure enough for webmail, so this article is going to look at adding HTTPS access to webmail.
My mail server runs qmail. This is the first of a multipart series showing how I went about installing vpopmail for virtual domain support.
In earlier articles, Displaying Progress in a Wizard and Cancelling Long-Lived Tasks from a Wizard, I discussed how to run a long-lived task from a wizard and how to display progress in the wizard.
This article, a follow-up to this one, shows how to modify our project to allow the user to cancel the long-lived operation.
The WinInet functions allow an application to interact with Gopher, FTP and HTTP servers. This article shows how to use the WinInet API to download from an HTTP server.
Quick tip: How to escape HTML text so that it can be quoted in another piece of HTML, e.g. in a <pre> tag.
How to write a CGI script (in Ruby) to accept files uploaded with POST.
I just had a strange error message from Visual Studio.NET when attempting to add a handler for WM_DESTROY to a C++
dialog class. It said:
Note: This article applies to Visual C++ 6. There might be an easier way to do this in Visual Studio .NET
Nothing particularly strenuous: Just handle NM_CUSTOMDRAW and play with clrTextBk. The only real surprise is that you have to return CDRF_NEWFONT from your handler to get the new colour used.
I just spent the best part of a morning chasing this one down, so I thought I’d share it with you:
Question: Everyone “knows” that you should write a decimal to Roman number converter using a table. What happens if you try it test-first?
Here’s some C source code that does it.
Provides a quick way to find out where Windows is putting your “Favorites”, “Application Data” etc. folders.
Our MFC application has a resource script. This resource script suffers from a minor problem: It’s not dependency-scanned. If we edit any file included by it – for example the .rc2 file, it’s not rebuilt properly.
CPropertySheet provides the SetWizardButtons function, allowing you to enable or disable the “Back” or “Next” buttons. It doesn’t, however, allow you to disable the “Cancel” button.
empeg’s source tree has a directory called lib, in which the core libraries used by all of our products live.
Unfortunately, this conflicts with one of the included pseudo-targets that jam uses.
Jam imposes a hard limit of 996 characters on command lines when built on NT. This limit is higher for other operating systems, and can actually be raised to around 10Kb on Windows 2000. However, it’s still not high enough for some link actions.
First, you’ll need to download jam version 2.3.2 from ftp.perforce.com in zip format or as a tar.gz
I’m adding a wizard to the program that I’m currently working on. The wizard walks the user through importing some information from a file. I’d like to be able to display the import progress as a seamless part of the wizard.
I was adding a new wizard to the program that I’m currently working on, so I copied some code from a Wizard97 demo program that I wrote some time ago.
The program that I’m working on at the moment needs a wizard to walk the user through something. I copied over some files from a Wizard97 demo project that I wrote a while ago. It all seemed to be going well.
Welcome to differentpla.net, where Roger occasionally posts snippets of code and other stuff that he feels like sharing with the world.
First, you’ll need to install a stock NFS server. On Debian, that’s as simple as apt-get install nfs-user-server.
There are at least two reasons for using FindWindow.
The ON_COMMAND_RANGE and ON_UPDATE_COMMAND_UI_RANGE macros are useful when you want to treat a group of commands
similarly. In this case, it’s the commands for changing list view style. The command IDs must be contiguous, and you
must specify the lower one first.
(Seen from the bottom of the connector, so that the tab is facing away from you, and the gold pins are facing towards you) Pin 1 is on the left. Pin 8 is on the right.
There’s some discussion of this in Microsoft Knowledge Base Article Q141751.
The reason that our previous example consumes 100% CPU is that it busy-waits while pumping messages. The correct answer
is to run the dialog box modally. However, we can’t (easily) use DoModal, because this causes ordering problems. Do we
create the thread or the dialog first?
One simple way to run the task in the background, while reporting progress, which still allows the dialog to be repainted/moved correctly is to busy-wait for the background thread to finish, while pumping messages:
We can solve the problem of the memory leak, and of the premature destructor call, by simply blocking until the thread is finished:
We add another button to the dialog:
Because we have progress reporting, we can make use of a progress dialog to partially solve all of these problems. In Visual C++, select “Project|Add To Project|Components and Controls”.
We can deal with the first one (no feedback), by using an hourglass:
Initially, we’ll run this as a foreground task. This will ensure that the code compiles and works before we get too complicated. Also, it’ll show why we want this task to run as a background thread.
```c++ // task.h class TaskObserver;
MFC provides support for automatically switching menus when the active MDI child changes. Why doesn’t it automatically switch toolbars?
It’s not easy to write a Windows program that takes a long time to do something and yet still seems responsive to the user. Consider that you’ve got a single task that needs to be finished:
Some code.
You can send the empeg player commands using the serial port. This is a list.
How many times have you found yourself writing something like the following?
Debian provides a powerful utility called make-kpkg for compiling your own custom kernels. It integrates tightly with
dpkg by making .deb files and you can even use this to prepare your kernel image on a faster system. Later on it will
make complex operations, such as compiling in patches and modules, a snap.
Note: This is a preliminary version of this document. I’ve not fully verified that the stuff in here works.
Because the DVD-ROM drive supplied with the SRX87 is Firewire, and is not supported by the standard Debian boot images, you’ll need to use network booting (using PXE) to get it going. This page has some instructions for installing Debian in this manner.
To get the DVD-ROM drive working, you’ll need to compile your kernel with the following options:
Note: Caoilte wrote this. I’ve not had a chance to check it yet.
First, you’ll need to recompile your kernel, with the following options:
You’ll need to install the ACPI patches, and the ACPI daemon. Grab the patches from here, and apply them to the kernel:
A page that I found recommends installing the spicctrl and sjog packages to get control over the LCD brightness and JogDial.
$ find . -type f -print0 |
xargs -0 md5sum > MD5SUMS
$ sort < MD5SUMS | uniq -D -w 32 > DUPLICATES
Grab a copy of rsh-client from the debian archive.
Some stripped down utilities for the empeg:
By default, AppWizard-generated applications use separate directories for the output of the different debug and release builds. We’d like to replicate that functionality.
Since the Audio Receiver Manager (the server) uses HTTP to stream the MP3 data to the Rio Receiver, it’s possible (with some work) to stream directly to Winamp. Here’s how.
Note that this only applies to v1.x car player software. v2.x uses a different error numbering scheme.
emplode v2.0 adds the ability to search your music database. It provides two methods of doing this: simple searches (which are hopefully so simple that I won’t explain them here), and advanced searches.
Something like this:
This morning, I decided to buy myself a scanner. I’m planning on using it for document filing – keeping my credit card statements on CDROM, etc.
While getting our MFC application to build with Jam, we got to a point where it would compile and link, but wouldn’t run. This calls for a trip into the debugger. The simplest thing might be to just run the executable within Developer Studio.
I just came across a problem where IEnumFoo::Next was returningRPC_X_NULL_REF_POINTER. Of course, the first thing that I did wasundertake a search in MSDN, and with Google to see if I could find anyexplanation of what the problem is.
There are times when you want to have multiple input files in a Visual C++ project configured to generate the same output file. For example, you want to catenate a bunch of files to generate a file to be #included.
The problem here is that you have a piece of IDL like this:
The default Jambase file doesn’t come with a rule to make DLLs (or shared libraries in the Unix parlance).
This information comes (indirectly) from TMC UK.
A while back, I was trying to get a computer built out of some spare parts to run Linux in the corner of the office. However, I had some trouble with random lockups. I think it had something to do with the memory configuration of the motherboard.
A couple of weeks ago, we got an LED message board at work. They had them on offer at Maplin (if I recall correctly). Anyway, the manual with the board is singularly useless. It’s labelled “Moving Message Sign”, and has a picture of the display with “Taiwan Kingpul” on it.
Recently, while wandering along to my local Debian mirror, I was surprised by the fact that it decided to come up in an Asian script.
Microsoft’s knowledge base article, Q179907 explains how to use a transparent CAnimateCtrl in a CView or a CDialog.
Having been so impressed with the speed of Linux on the A7V (see here), I accidentally slipped while in Cambridge Computers and bought another one.
As mentioned here, my Pentium II-400 got fried by an ocuk power supply. As they say, however: Every cloud has a silver lining. With a dead computer, I could finally persuade my girlfriend that some long-overdue upgrades were necessary.
I don’t know if there’s something in particular about Overclockers.co.uk, but we’ve not had any good experiences with their power supplies. We’ve had two power supplies from ocuk, and they’ve both gone horribly wrong, and fried a whole shedload of hardware.
Warning: I’ve recently received a couple of emails stating that this technique totally breaks Web View in Windows Explorer. Use it at your own risk. Back up any files you change.
This one’s very rare. I don’t anticipate it ever happening to anyone else. It did, however, happen to me, so I thought I’d share it.
My name is Roger Lipscombe.
Senior Software Engineer at Happening, the tech powering Superbet.
I blog about programming and other technology and gadget stuff.