arbitrary user

Jonathan Endersby, Recovering Technologist

Archive for category Tech

Quick and Simple Server SMTP

Sep 15

Posted by arbitraryuser in Linux, Tech, hacking

I have a number of servers that I look after in various places on the intertubes. I like to have things like MDADM (Linux software RAID manager) be able to mail me when the something goes wrong like a disk dies etc.

Some of these machines are in places without reliable SMTP servers for me to send mail through and I’ve tried running my own postfix and delivering the mail directly, but invariably I run into situations where the servers that I’m trying to deliver mail to don’t like DSL IPs… and not getting a mail about a dead disk is kinda a big issue.

I also don’t trust a lot of ISP’s SMTP, and some of my servers move around, so one day it’ll be behind a DSL IP and the next behind a Verizon IP (where it can’t talk to smtp.dslprovider.net etc).

My solution is quite simple, use google. (This guide is for Ubuntu but I’m sure you’ll figure it out with other distros)

  1. Create a gmail account for monitoring. I do this because I don’t want my gmail password floating around in plaintext on various machines.
  2. Install the ca-certificates package

    $ sudo aptitude install ca-certificates
    $ sudo update-ca-certificates

  3. Install msmtp

    $ sudo apt-get install msmtp

  4. Configure msmtp

    $ sudo vim /etc/msmtprc

    Set it to something like

    account gmail
    host smtp.gmail.com
    from myemailaddress@gmail.com
    auth on
    tls on
    tls_trust_file /etc/ssl/certs/ca-certificates.crt
    user notifyemailaddress@gmail.com
    password mys3cr3tp455w0rd
    port 587

    account default : gmail

  5. Create a sendmail simlink

    $ sudo ln -s /usr/bin/msmtp /usr/sbin/sendmail

  6. Run a test

    $ echo “This is a an awesome test email” | msmtp youremail@domain.com

  7. If you want mdadm to mail you when something goes wrong

    $ sudo vim /etc/mdadm/mdadm.conf

    and put your email address on the line that reads something like

    MAILADDR youremail@domain.com

  8. And then run a mdadm test by running

    $ sudo mdadm –monitor –scan –test –oneshot

  9. If everything is working according to plan you should receive an email. You can now rest assured that any future MDADM issues will get to you.
1 Comment

Visualising the Interest Rate

Mar 24

Posted by arbitraryuser in Aggregate This, Property, Tech

I though it might be interesting to try and graph the Reserve Bank’s prime rate data… It goes back a long way. I used Python to scrape and collate the data and PyCha to generate the graph.

UPDATE: I’ve replaced my graphs with new versions made by Russell who corrected my original code by interpolating the data correctly over the y axis.

This is the narrow version.

And this is the wide version (click to download the actual 10000px wide png)

Interestingly enough, todays rate cut *was* on that page earlier today, but now I see it’s gone… so I inserted it manually ;)

No Comments

There is no cure for stupidity.

Feb 11

Posted by arbitraryuser in Aggregate This, Tech

A while ago I blogged about a weird comment I had received on one of my blog posts.

In summary, there is an SEO company called SEO Results (aka BizSearch, aka NetAge) that gets its staff to trawl blogs and write comments with the Author URL set to the url of one of their SEO clients.

Author : PMM (IP: 165.146.34.239 , dsl-146-34-239.telkomadsl.co.za)
E-mail : kim@bizsearch.co.za
URL : http://www.pmmproperties.co.za
Comment:
Wow what a difference it looks fantastic, great job done

One would think that after the first run in I had with these spammers they would have avoided my blog?

Anyway, to make sure it’s clear: SEO Results are spammers and black hat SEO idiots… Using them is likely to get you bad mouthed on the internet (like this) and perhaps worse, blacklisted on google.

5 Comments

Eye Witness News (ewn.co.za) has a few issues.

Jan 20

Posted by arbitraryuser in Aggregate This, Tech

First let me say that I like the idea of a new, fresh news site… EWN could quickly become a serious player in the news arena, but before they do so they’re going to need to fix a few issues.

I sent an email listing some of these issues to the Primedia team. I know it got there because people who know people said there was some flapping and urgent updating that happened as a result of the email… However, I’m yet to get any form of reply whatsoever… which I think is just rude.

(update: A few things (like the comments about Mandela) have been fixed, but the overwhelming majority is still as it was when I wrote this list a few days ago. The site however seems to be suffering from lots and lots of timeouts now.)

This list is by no means exhaustive…

1. You need to add a DNS record for ewn.co.za (so that http://ewn.co.za actually works)

2. You need to add RSS, preferably ATOM, with a number of sub feeds, geographic locality etc.

3. You need to remove your stupid comments from your html source… not only is it dumb, but people WILL take offence.

<!–<li><a href=”#”>Mandela Gives Birth to a Gorilla </a><span class=”timeadded”>2&nbsp;days&nbsp;ago </span> </li><li><a href=”#”>Prengant Child attacks Mandela</a><span class=”timeadded”>3&nbsp;days&nbsp;ago </span></li><li><a href=”#”>Tourists Can’t Give Enough Birth </a><span class=”timeadded”>1&nbsp;day&nbsp;ago&nbsp;</span></li>–>

etc

4. You need to make sure all your templates actually work… for instance this one is a little too concise –
http://www.ewn.co.za/story.aspx?id=4013

5. You need to protect yourself from SQL injection and handle any attempts gracefully.
ie. http://www.ewn.co.za/articleprog.aspx?id=40%2709

6. You should probably consider looking into better urls for your articles, specifically for SEO purposes.

7. You should also probably add meta descriptions (and possibly tags) to your article pages. This will help display relevant content in search engine results.

8. Your pages do not even come close to validating XHTML transitional.

9. You need a mobile version! This is easy to implement!

10. That logo… It’s very 90’s.

11. Bonus Tip: One of my biggest gripes with the other news sites is how they never allow you to view larger versions of their images. Implementing Lightbox2 over you existing site will be easy and help
differentiate yourselves from the other players.

12. Your site search is broken in Firefox and Safari and is unstable in IE6 and 7.

13. Your server errors (timeouts etc) need to be handled more gracefully. At the moment your site displays the default .NET error pages, which is something that only the developers should be seeing.

14. Your comment form gives no indication that it hasn’t submitted due to invalid data. This will confuse users.

15. Besides the SQL Injection issues, users who search for any string that contains an apostrophe will be greeted by a rather ugly error page. Try search for o’grady.

16. You need to remove all your test data from your database. http://www.ewn.co.z/articleprog.aspx?id=183 etc

17. You should add a clearfix after your pull-out-quote on your article pages. This will ensure that articles that start with single character words like “A” don’t end up displaying the first character to the right of the pull-out with the rest of the article below the pull-out. See http://www.ewn.co.za/articleprog.aspx?id=4021

18. Your logo should be a link to your landing page. This has become a web standard and a lot of users will expect it to do so.

19. You should sanitise your article source before your editors submit it so that you don’t end up with styling imported from MS Word which can break your layout. ie. 

<p class=”MsoNormal” style=”MARGIN: 0cm 0cm 0pt; LINE-HEIGHT: 12pt; tab-stops: 18.0pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt”>

Not only is it ugly but it will repeatedly break your validation.

eg. on http://www.ewn.co.za/articleprog.aspx?id=4033

20. While it’s debatable whether this is a true bug, there is a fair amount of functionality on your site that is broken when the user disables javascript.

21. As I’m browsing the site I am hitting a lot of timeouts. This indicates that your server is probably struggling. Most likely due to bad coding and/or a database that isn’t tuned properly.

22. Your cache control is not good. You should probably add far future expire headers to all your static resources. This will speed up the site for regular users. Also, combine and gzip your js. This will also decrease load on your site and help with all the timeouts.

5 Comments

Girls and XHTML Validation

Jan 13

Posted by arbitraryuser in Aggregate This, Tech

If you’re ever debating whether or not something is sexist, change the gender statement into a racial one and see how it fares…

ie. (taken from the intertubes)

Lucy
We don’t know a whole lot about Lucy, except that she’s one of the few females on the planet who can hold a conversation about search engine algorithm changes and validating XHTML pages.

Changes to:

Sipho
We don’t know a whole lot about Sipho, except that he’s one of the few black people on the planet who can hold a conversation about search engine algorithm changes and validating XHTML pages.

12 Comments

The coming revolution…

Dec 17

Posted by arbitraryuser in Aggregate This, Life, Philosophy, Tech

The internet, for all it’s vices, has made the world an incredibly small place. I buy books from Seattle, business cards from London and storage space from San Diego. I chat with friends in New Zealand, India, Finland and Joburg on a daily basis. At ground level the online economy seems pretty stable. Online businesses are lean, mean, fighting machines forged in the dot-bomb furnace. Compared to the sumo wrestling auto industry we’re Ethiopian long distance runners. (Enough with the analogies now)

The offline world however is in a crisis, big corporations are falling over on a weekly basis… most of them failing due to fat cat, short sighted management, while others are just innocent victims of the carpet bombing that is this economic train wreck.

Then I read things like this. George Oates, one of the key people and designer at Flickr, got let go by Yahoo, who bought flickr a few years ago. It’s not so much the fact that they let George go, but rather they way they did it… Basically getting her manager to call her while she was overseas and read a message to her from a scripted “cheers” letter. Her blog post about the ordeal is brutal. Within 14 hours of the call she had lost all her privileged access to all that was flickr; something that had been the centre of her life for many many years.

George’s story is the logical conclusion of the ‘corporatised’ world that we’ve all bought into… and I think the world is starting to see the folly in supporting a system that can turn around and kick you out when you least expect or deserve it.

I have this sense that people are starting to dislike, and distrust, big corporations. In the 50’s and 60’s corporations were the saviours of the failing economy, hell, if you could work for a corporation you were sitting pretty… Working for a corporation meant you had a stable job and even though all you got for 50 years of service was a hundred dollar watch (who needs a watch when you’re retired anyway?), you were happy to have had the job.

But the world is different now, for whatever reasons people expect more from life than just ‘having a job’. We want to have fun, be challenged, enjoy working, laugh, be successful and get home on time to have make supper for our smiling kids and watch 30 Rock on Tivo.

So where does this leave the workforce? Well, the internet is making *not* working for a corporation easier and easier. Now days your small print shop in a side street of London can turn into an international brand with customers from Tibet to Texas, but, most importantly, that small print shop doesn’t need to become a overweight corporation in order to carry on being successful. It’s the long tail global customer effect. Hell, you could sell clothing for conjoined twins on the internet and still swing a profit.

Perhaps more interestingly though, the internet has made running your own company a lot easier. Re-read that last sentence. The internet has been around for almost a gabillion years now, but it seems like only in the last 5 years has the promise of “running an online business from your garage” come true.

Perhaps the supreme irony of the situation is that Yahoo itself was once a small company that got big, and in turn bought up flickr, the blood, sweat and tears of a small team, most of whom have subsequently left Yahoo or been fired. How different life would have been for all those people who gave birth to flickr, if they’d just stayed a small team who focused on being the best and staying happy while doing it…

Corporations have been holding the workforce hostage… but the distributed client base and self organisation of the internet is starting to make it harder and harder to not start your own thing, or join a small company with big vision.

Similarly customers are more and more looking for micro providers, buying local produce, supporting up and coming manufacturers and looking to identify themselves as unique by buying products that weren’t made in batches of a million. Perhaps it’s the inherent knowledge that the companies that are producing t-shirts in batches of a million are run by the same kind of people that will fire you from the very company you helped start and feel nothing while doing it.

You’re a person… let the machines be the robots. The revolution is coming, and it won’t be televised, it’ll be broadcast.

1 Comment

Plasma Rockets and Awesome Comments

Nov 26

Posted by arbitraryuser in Aggregate This, Tech

Sometimes the comments are better than the content…

ricewookie: uhh fan?

rmessenger: No, this is a prototype VASIMR engine in a vacuum chamber. This could very well take people to Mars in less than a month. Even take humans to Jupiter’s moons and the outer solar system. It’s not a fan.

Read the wikipedia article on VASIMR rockets.

No Comments

Clickthinking ClickJacking

Oct 9

Posted by arbitraryuser in Aggregate This, Tech

I love a good game of internet pile-on as much as the next guy, but god damn this is hilarious.

The brilliant designer Coda, based in Cape Town, regularly gets his site design jacked by punks all over the world and most of the time he just laughs it off. Then the other day someone pointed him to the new Optimal Energy site done by Clickthinking, a Cape Town based (ie. They must know who Coda is) “web company”. You can read Coda’s opinion here.

Right now I guess Clickthinking are busy digging the hole they plan to live in for the next few weeks until this quietens down, but damn, Optimal Energy should be pissed… They got fleeced and are probably feeling pretty damn uncomfortable about their *brand new website* right about now.

ps. No link-love for Clickthinking… just google them.

1 Comment

Again and again and again

Oct 6

Posted by arbitraryuser in Aggregate This, Tech

So after chatting to some people who love their hacked iPhones, and seeing a terminal running on one, I have to admit that they’re very pretty and possibly very powerful. However all that power is being locked up inside the shiny little box, and Apple seems convinced that that is the way forward for their platform. They’ve even resorted to telling some app developers that their apps are too similar to the existing Apple apps, and therefore won’t be allowed onto the marketplace.

But before I turn this into an anti-iphone blog, the reason I’m again saying that Android is going to eventually reign supreme is because Motorola (whose cell phones I’m not a huge fan of) are planning to employ 300 developers to work on Android internally. While this might just mean that Motorola ends up producing more crappy cell phones, it also might mean that they are having a bit of a mind shift… (possibly caused by the iPhone’s success?) and want to build something that competes for that power user rather than the 16 year old school girl they’ve apparently been developing phones for for the past few years. And physically motorola build solid devices. They have a rich history of building miliatary grade equipment and comsumer trendoid crap like the Razr (Is that how you spell it?)

Regardless of what Motorola end up doing, the real winner is going to be the Android software stack that they’ll no doubt push code back into. I’m not sure how their licensing is going to work, and they might chose to close source all their android applications, but they’ll no doubt have a positive effect on the Android operating system/software stack, and that’s all that matters.

ps. Thanks to Jonathan Carter for pointing me to the news story.

2 Comments

The joys of a non-tech girlfriend.

Sep 24

Posted by arbitraryuser in Aggregate This, Life, Linux, Tech

Lynnae, as you may know, is a food nerd. She teaches me stuff about how broccoli is from the Brassica family and therefore not suitable for stocks if you want a clear stock and that toast smells nice because of the Maillard reaction. So she’s a nerd, but she’s definitely not a computer nerd and it’s kinda funny when she’s trying to understand what I do all day or tell me about some or other computer problem she has at work. She has a windows box and a mac… no prizes for guessing which one “flashes on the one window and then the other one goes orange and starts flashing too and then you click on the tab for the first one and it starts flashing too so I had to work the whole day while the screen flashed at me“.

The real gems seem to pop out of nowhere:

Who pays for the network waves?
- Asked while I was trying to explain the concept of a wug.

Maybe it’s a glitch.
- Pretty much anything that goes wrong is “possibly a glitch”.

Something with a megabyte.
- When asked what size the SD card in her phone was.

What’s an aggregator, is it an angry alligator?
- Ok, admittedly she’s trying to be funny.

But she’s learning… We have a shopping list wiki, which she thinks is arbiwikiwiki.com (or something) but she runs firefox and is starting to understand why open source is better… that, and she’s the most awesome friend I’ve ever had, so I’ll tolerate her noobness.

2 Comments