slef-reflections on Webmastering
- eCommerce and Online Retailing
- Feedback and Comments
- Libraries on the Web
- Search Engines
More cut-and-paste browser fun: message-id search engine
Posted by mjr 2007-10-30
If you use a firefox-based browser, bookmark http://mid.gmane.org/%s then right-click it in your bookmarks menu, pick Properties and enter "message-id:" in the Keywords field. Then you can paste Message-Id lines from email headers (or some debian list posts) into your browser window and it will search gmane for it. Handy.
Didier Raboud comments:
"It even works (for a long time...) in Konqueror..."
Thanks for letting me know! I also forgot to mention that I learned of it from Eduardo Habkost
ASK 3D - New but not good enough
If I search for something in the UK, it switches back to World without asking (I suspect some undisclosed cookie-fu). If I search for certain company names, it auto-corrects the spelling and returns a page of irrelevant results. The results pages seem to have too much space occupied by adverts. And the list goes on.
I'm ready to switch search engines, but it won't be to this version of Ask 3D.
Sustainability, Search Engine Traffic and free software webmasters
2007-06-29: Regular readers may remember that I'm a vocal critic of some search engines and the unreliability of their fast-changing algorithms. One day you're top ten and the next you're gone. If you're building your business on search engine referrals, you're building your castle on shifting sands.
"one of the best moves you can make for the sustainability of that site is to lower the percentage of traffic that comes from search"
That's one reason why I'm glad to see the suggestion of etbe: Advertising Free Software Projects - nearly every free software project would be better off improving their web sites, adding cross-links and building more sustainable traffic, instead of wasting resources competing in the Google-ad Games.
Unless you've got the thousands of pounds a year that successful AdWords marketers spend, your Google-ad success will probably be short-lived, as people with more money will outbid you for keywords that work. It's even less sustainable than search result traffic, because it's consuming your financial resources as well as your vounteer time.
Meanwhile, most free software project webmasters I know are over-stretched, but many wannabe-contributors seem to want to nuke the entire site to rebuild it with their pet tool. Those offers are really about their ego more than helping the project, in my opinion. They want to Get Themslves Noticed, but I've seen far too many unfinished site rebuilds to like those offers.
In fact, I don't think any good webmaster would offer a rebuild without the site owner asking for it - even in the rare cases where I really strongly want to restart a site, I explain why and ask permission first, not just wade in with an offer to nuke-and-rebuild. If the site owner says no, I move on: there's no shortage of work here, after all and very few free software projects pay their webmasters.
So, if you want to help a free software project, ask their webmasters for suggestions and access to the site source code.
What is needed to start Open Social Networking?
2007-09-25 (Permalink): Since the closure of SoFlow and adding myself to LinkedIn, I've been thinking a bit about Professional Associations and using Social Networks (SNs) for business. I've not worked out my answers, so this is a "dear lazyweb, what pieces of this jigsaw am I missing?" post.
Much of the recent mainstream news has been about the legal problems of SNs - for example, see BBC: Web networkers 'at risk of fraud' (which was also in items on a recent Click show on BBC World and News24) and BBC: Facebook site faces fraud claim - or about how they could Save The World, like BBC: Social net offers new perspective
I think there's a bigger problem which will stop SNs making a deep-change to online activity. Each time a SN like SoFlow closes, it wastes some resources of its members. I'm starting to side with other developers in being tired of seeing my input turn to bitrot, through buggy software or buggy business models. I'm not alone: see Gunnar Wolf: On social networks for example.
A good idea seems to be on Slashdot: It's Time for Social Networks to Open Up but how? Each SN seems to guard its dark corners as vital elements of its business model. Should we look to our Professional Associations to create new Open Social Networks (OSNs)? After all, SNs aren't the core money-makers for professional associations.
But, the biggest problem with Professional Associations is picking a trustworthy one - one which will do what it says on the tin, instead of taking your subscription, then undermining you. Consider the British Computer Society and its OSSG License: how many of its members really wanted another software copyright licence? Fortunately, it was spotted - Volunteer brickbat thrower needed in London (Tuesday 24th 6pm) - and the Report from BCS OSSG meeting on a new FS license suggests the negative feedback was loud and clear, so hopefully we won't see that again. If I was a BCS member, I'd probably not renew after helping to fund that harmful activity.
librarian.net: choosing your battles and choosing your professional associations talks a little about some library associations as a sort of case study which many will recognise as familiar, but there's an interesting comment from Blake there:
"I think there's an association out there for everyone."
Is there? What association is out there for liberal hackers and webmasters? I don't think I've found it yet.
So, should we be looking at ourselves, rather than existing professional associations? Can we do Open Social Networks peer-to-peer in a meaningful way? Are we already doing it with blog feeds? How should we expand on that?
Although I disagree with the conclusion that using another Social Network site with an uncertain business model is How to succeed with social networking, the article also mentions that
"social networking is in its infancy and that it will not really get underway until we have portable profiles and interconnectivity"
In other words, it's our revolution - let's steal it back.
"debian comes close"
That idea appears periodically, then discussions like this one from March 1999 kill it for a while, so I'm not sure.
Ben Francis commented:
"I wrote some notes on this a while back.
I think the key piece to the puzzle is writing better tools for FOAF (and other related standards) and promoting it. Most other features of social networks can be boiled down to emails, blogs, calendars and contact information for which there are already widespread open and distributed solutions.
It would help if there were social networking sites like Facebook and LinkedIn which used these standards rather than their current closed state. This would allow people who don't want to run their own software to connect with those who do and allow inter-network interaction. At the end of the day the web is the social network, the idea of everyone in the world having to sign up to multiple proprietary services to "network" with other people is extremely daft."
I'll take a look and try to rewrite parts of my sites into FOAF soon.
Putting the Tour de France commentary on your root window
#!/bin/bash # keeps updating le tour on the top left of the root window # Depends: bash, netpbm while true ; do xsetroot -bitmap <( curl -s 'http://live.cyclingnews.com/?id=latest' \ | sed -e 's/<br[^>]*>/\n/g;s/<[^>]*>//g;s/ / /g;s/ *$//;1,/Tour de France/d;/\(rovisional\|start.*prev\)/,$d' \ | cat -s | fmt \ | pbmtext -builtin fixed | pnminvert \ | pnmflip -tb | pnmcut -h 620 -pad | pnmflip -tb \ | pnmpad -black -w 1024 -h 768 -b 620 -l 0 \ | pbmtoxbm ) sleep 300 done
This was linked from netsensei
"pamflip does not seem to be in the fiesty netpbm package nor does it seem to show up in packages.ubuntu.com."
Huh? Ah. Ow!
Once again, I fall freely into a freedom freefall.
It seems that netpbm is not the same as debian's netpbm (also used by ubuntu) and the machine providing my root window is currently using netpbm rather than debian's one. (Why? Not sure yet. It annoys the hell out of me, because every man page refers to a web page.)
Apparently debian has its own netpbm because
"Upstreams and debian goals differ in some aspects too much to build the package on upstream versions"
How? In what way? No further explanation is linked. Anyone got a good intro to the history of this?
Any road up, it seems that debian's netpbm doesn't (yet?) have pamflip, but it does have pnmflip. I've changed the script to match.
"Often, dustups like these are the result of upstream not caring about one or more of the following:
1) Licenses. Many people will release code with no license attached to it at all, which means (Thanks to the wonders of the Berne convention) that no one else can distribute it.
2) Patent restrictions (the gif patent issue is now dead, but there's still patents on some other graphic formats)
3) Restrictions against selling the software for commercial gain, using the software in the military, in biotech, etc.
The netpbm copyright file seems to include instances of each of these. I don't know how it escalated into being unable to build based on upstream."
Thanks for the info. I'm still surprised debian doesn't keep a (maybe big) patchset rather than a fork on this one.
I don't remember what put me off it, but maybe imagemagick isn't so bad...
New (or new style) memberships and blog groups
2007-07-05: I was accepted as ODP Editor: slef for the Weston-super-Mare Business and Economy listings a while ago, but I forgot to mention it here until now. I've just done a bit of basic tidying there so far and I'm finding my feet. ODP seems rather more structured and process-y than WWWVL ever was. I'm not sure whether that's good or not.
I've done a bit of CSS-tidying on my main site, my debian site, and OtherWayUp - I'm also looking to launch new groups on OtherWayUp. If you have a blog from or about Somerset or Cooperatives that you'd like to appear there, please leave me a comment or email.
How fat is your feed?
Because of the problems with my aggregator's memory use, I'm unsubscribing from some feeds that are too big, too slow and not vital enough that I'll miss them. So, it's goodbye to:
no2id.net newsletters - too big (144k), too slow (dates back to February) and not vital - I'll keep the events and news release feeds for now.
www.thewildbeast.co.uk - too big (72k) and too slow (dates back to May 2006).
scobleizer.com - I subscribed to this after a recommendation, but it's too big (128k) and actually rather irritating IMO, waffling about things that I'm not interested in.
www.sbs.gov.uk - site seems to have vanished without warning. Typical failure of our government to make life easy for small businesses.
aphoc.org - site has been offline too long and I don't remember what it was.
There's a couple more which I'm keeping under review, but they've earned their keep for now. (Update: removing just the above has shortened the merge time from 3mins to 30secs.)
My feed is about 29k at the time I write this: how big is yours? If your feed is larger than about 50k - particularly if you don't send a Last-Modified or Content-Length header - you should take steps to slim your fat feed. Consider reducing the number of items, particularly if you have stuff more than three months old in it.
Am I lazier than most at unsubscribing from feeds - my main feedbox still contains more than 70 - or will people with fat feeds already have lost lots of readers?
"perhaps the ultimate example of this is the ebay uk announcements feed - the items date back to 2004 and it totals 267.86 kB - so big that the livejournal syndicator refuses to use it at all. I went so far as to contact ebay about it, but they didn't do anything."
I'm having a little problem getting some money from ebay (which I might write about after I resolve it), so the above comes as no surprise to me. That sounds like the Carlsberg of feeds: probably the fattest feed in the world.
"I think a feed should contain all content the other versions of the page have, it's an alternative (link rel etc.) after all.
Imho, parsers should just be made smarter, and drop things they consider to be 'too old'."
I agree that a feed should be the same content as other versions (or at least the main one of them should... see my site's filtered feeds) but my comment about being too big and too old applies to the non-feed versions too: Recent Changes pages should drop items which aren't recent.
I disagree that parsers should drop things that are too old because it would do more harm than good: generally, you can't tell how old an item claims to be until after you've read it; and a lot of publishers have subtly broken clocks (which is why my aggregator uses ordered lists, instead of sorting by timestamps like planet).
Related link: Problogger: 34 Reasons Why People Unsubscribe from RSS feeds.
British Library Blogs
After noticing the Library of Congress Blog (thanks for the anonymous link correction) via librarian.net, I asked the British Library and they told me they had a blog for the London in Maps exhibition and will soon launch a new content blog called Sacred. Which other libraries are blogging?
"One blog that spans the divide between paper-bound librarians and the digital world is the Disruptive Library Technology Jester
I found it as we both are working with a piece of software called 'Fedora' (not the RedHat distro, but http://www.fedora.info/ an interesting open source XML-centric file repository software.)"
Feedback and Comments
Media-sponsored Forums - How to Kill a Community
2007-09-19 (Permalink): I was asked for a story on good or bad media sponsored online conversations. I'd not really had much to do with them until this summer.
itv is the UK's biggest commercial broadcaster and it had rights to show the Tour de France here this year. Its coverage was pretty good, with daily highlights and live coverage at weekends, mostly on the itv4 satellite/DTT channel. The end of every show plugged their web homepage, which had a prominent link to their forums.
To cap it all, itv closed the forums for a weekend for an upgrade. Then a weekend became a week, which became a month, which became six weeks. When they finally reopened it, at least they sent me an email, so I tried to login. My login doesn't work and I can't view the forum as a guest any more.
I've emailed their support address and had no reply. I don't know if this is a general problem - I can't contact other forum members because I don't have their details. I only saw them on the web forum and members couldn't link their homepages.
So, it'll be a long time before I spend much time participating on any media-sponsored sites again, except in a casual throw-away way. Now most of my sports messages will be posted to Cycling Fans and BuzzinFootball like my local news posts go to the community-run WSM Forum. The trouble with media-sponsored forums is that they just don't have the interest in any particular community to want to sustain it. Somewhere, there's a bean-counter saying "so what if we kill this community? We started it from scratch and we'll just do it again." Any messages you post to those forums are usually gone, out of your control, lost to you. Is it worth posting to them?
I think Andrew Savory is also spot-on about the problems with forums for technical communities in Bagel Belly Blog: How Open is your Open Source project? He certainly seems in fine form so far this week.
I know not all forums are that bad. I'm on a couple of phpbb-powered ones which work well, with added XML feeds and so on. Are any "old media" companies brave enough to run a liberal phpbb? Are there other examples of particularly good or bad forums that you'd like to tell me about?
Commenting on blog posts
When I have a longer comment on someone else's blog post, I usually write it up on my own site and then send a pingback or trackback. I've written my own really basic pingback and trackback clients and servers, which seem to work. I'll package them up Real Soon Now. Sooner if anyone asks for them.
Update: Cord Beermann commented:
"If you have a problem with the trackback function of my blog, then please be more specific what is wrong with the trackback function, because the software may be part of Debian soon."
trackback:ping="" which seems obviously wrong.
"The 'stupid eyetest' however is a personal decision for my blog."
So what if it's a personal decision? It's stupid and I'm going to say that it's stupid. Are you the sort of guy who takes a personal decision to kick guys in wheelchairs, but then protests if they run over your foot in return? I'm disappointed if more stupid software is going to be in debian.
Update 2: the image test seems to have been replaced by some numbers now. Thank you for removing the eyetest, but it still won't save a comment for some reason.
"However: both isn't related to the initial problem."
I think the broken trackback/pingback support is related to your initial problem. If your trackback/pingback support worked, you could use a real editor on your own site instead of a web browser on someone else's.
In other news, here's Alexandre Vassalotti's review of Emacs weblogger-mode.
Working feedback routes attract subscribers
As I mentioned before, I'm getting quite harsh in unsubscribing from buggy feeds. I'm also being quite harsh in not subscribing to sites which have broken feedback systems like no comment box or email address, broken (rather than non-existant) trackback/pingback support, eyetests, or dumb tools like Spam Karma 2.
Therefore, I'm not subscribing to Limes & Lycopene, recommended by Brane Dump: The Thoughts of Matt Palmer, because although it's an interesting site it has a broken feedback route, which makes it less attractive. It's more than a bit annoying to want to interact and then be blocked by a broken site.
I still subscribe to about 100 sites, but that number will probably fall further. How do you decide what sites to drop?
"delete the feeds you find yourself reading less and less frequently"
In general, maybe, but there are always exceptions, like feeds which rarely update but carry important information when they do. I guess I should scan my subscriptions to see if there are any extremely unupdated feeds.
Update: tm clarified:
"I meant those feeds you've subscribed to, but find out you don't read so often, regardless of how often they're updated. for me, one of those is/was: http://feeds.howstuffworks.com/DailyStuff"
That's not so easy for me to tell. I read my subscriptions through a planet-like page, so they appear in the page flow whether or not I expand them or not.
Matt Palmer replied:
"I've got to say that I've never had a problem commenting on Limes & Lycopene. I suspect that the problem you had with Alexis Sukrieh's blog was likely unrelated to SpamKarma2 (or if it was SK2, it was a dodgy deployment -- you don't sound too sure what the cause of his breakage was).
On the other hand, if there is anything broken on a blog, the polite thing to do is to tell the owner about it -- it might not necessarily be a problem for everyone (although with SQL errors and things like that, it should be possible for the site owner to get error reports via e-mail, but how often do you reckon that gets done?). And yes, Captchas give me the faeces too.
For L&L, at least, I know for a fact that any problems will get looked at and fixed up damn quick. Contact e-mail is on Kathryn's site."
I sent a pingback. No response yet. I'll try email if there's no response later.
"Oh, and a comment on your comment system -- the text entry box is tiny (6 lines by maybe 35 characters just isn't enough for a literate comment)."
My preference is for 40x20 but if I make it that big, I get feedback that it's too big. It's always a compromise. It's meant to be about 22x8 now, but what size should it be?
Matt Palmer wrote:
"Aaah, OK. I have no idea whether [pingbacks] work or not; they're deeply in the black magic category -- never used them."
They're just a remote procedure call. Shouldn't be black magic at all, but a surprising number of blogs seem to do them brokenly, while supporting the more complicated (but just as spammable) comment and trackback systems. On size:
"40x20 would be a good size for me. I'm not sure why people would complain that the comment box is too big, unless perhaps it interfered with the good layout of the page."
I don't think it interfered with the layout. I'll increase the size again Real Soon Now.
Josh Triplett commented:
"For me, the size of the edit box doesn't matter much, because I don't use it. I just click the little "edit" button next to it that the Firefox^WIceweasel extension It's All Text provides, and then type my comment in Emacs. :)"
Well, everyone who lets web site scripts use their CPU and electricity... Thanks for all the helpful comments.
Update: Debian, and other stuff writes about A solution for a better spell checker in firefox : a better editor
Joey Hess commented:
"I'm suprised that noone has mentioned how to get a text area to fit the whole width of the browser window, no matter what width it's set to (I prefer approximatly 60 characters wide sometimes, and nearly 120 other times..):
Just put width: 100% in a stylesheet for the textarea.
Unfortunately I don't know a good way to dynamically scale a textarea vertically."
That's container-width, which is not window-width for my site, but I'll give it a go.
- Comment on this
- See also Blog Tools: Comment Tools
- Start of this section
- Start of this page
- Latest posts
eCommerce and Online Retailing
No-monthly-fee UK Card Payment Web Service?
2007-09-05 (Permalink): I want a way to take small, infrequent, one-off card payments on my web sites. Some of my customers much prefer to pay by card and I'd be happy to do that, if there's a decent card payment service. Is there one? I've not found it yet.
What do I want? The main thing is not to have a significant monthly fee. I'd rather not be paying if I'm not using the service, but I'd probably stomach up to 5% transaction fee for a good credit card handler. The 8% fee listed by Moneybookers seems a little steep, but I might use it as a last resort.
I'd prefer no automatic "inactivity" account closure if I don't use it for a couple of months. Sometimes I take no card payments for a while, then need to handle several at once. I'd also prefer a cooperative or mutual, but that seems unlikely in such a small field.
Paypal won't let me take my money out to my business account. To do that, I need to upgrade my account, but Paypal do not list LLP as a valid type of business and won't fix my bug reports by web, email or telephone. Last time I telephoned, after patiently explaining to them what an LLP is, I was told to put a different business type on the form. That scared me, so I asked for email confirmation of the instruction, which they said they'd send, but it never arrived.
NoChex froze my account without warning, due to inactivity. I've sent in their reactivation forms a few months ago and had no reply. I've also filed a web ticket today, but I've done that before. They don't even list a telephone number any more. That's a worrying sign in a bank.
So, both of these providers suck for the above reasons and I still need a way to take this month's card payments. All the UK banks charge monthly fees, don't they? Are there European card services which I should consider? International?
Jon Atkinson wrote:
"Have you considered Google Checkout? They process all the major UK cards (Visa, Mastercard, and Switch/Maestro).
Transactions with GC are charge-free until 2008 as they try to gain traction. I believe after that they won't be charging a monthly fee, but taking a small percentage on each transaction. These charges are waivered if you spend money on Google Adwords, which may or may not be of use to you.
However, you're probably correct in that all options are 'equally bad'. Google aren't doing much to differentiate themselves in this space, but they do bring a massive amount of brand trust to their service. I'm not sure about the accessibility status of their checkout procedure."
Martyn Drake commented:
"Dare I say it? Google Checkout. Might be a bit of an overkill, might not. But worth "checking out" nonetheless? Why all the questions? ArghhhhhHHH!"
"I'm loathed to suggest Google Checkout knowing your dislike of the operating company, but I've found it works reasonably well. Their web-based interface has a form for sending invoices, but it looks like you can also create links programmatically. There is no monthly fee, and commission is zero until the beginning of next year."
Terry Lane also suggested Google Checkout.
Joe Forster commented:
"There is Worldpay. A few hundred quid to setup and a few pence more than you would pay a normal merchant but guaranteed set-up and easy to integrate.
Google now offer a payment solution which is very good.
All of the above do not require an IMA as they either come with one or process payments through their own, similar to PayPal."
An anonymous (french?) comment:
"maybe amazon ? Their latest micro payment service sounds to be the one for this kind of thing and it is charged through their system.
all that is needed is people need to have an amazon account."
So, I've at least three more to look at now. I'll post reviews when I do. Any more?
Protx Updates Broke Shops
2007-08-15 (Permalink): Protx, a payment service, installed a major update on 1 August. They'd warned us about this in newsletters and we were prepared. We paid a bit towards another company's development of the protx_form module for OSCommerce to meet the updates and generally clean it up.
This went slightly wrong in three ways.
Firstly, protx went down for longer than expected, ultimately generating an apology from the MD. Not much we could do about that.
Secondly, protx started rejecting commas in amounts. GB uses commas in between thousands, so this meant none of our British-localised shops could process orders over £1000. Ow.
The Protx VSP Form docs still shows commas in the Amount examples, so I suspect that change was unintented. I've sent Protx Support an email to ask. I was offline, but Paul installed a fix before I got back.
Finally, the mainline protx_form module has updated to v1.17 to include these changes and more (see Payment Modules forum threads), but our developer hasn't uploaded their work yet, so we're stuck up a branch and someone needs to merge. I hope it won't be us, as I can already see conflicting changes.
What should I have done differently? Is it time to find a new payment provider? Should I insist that we publish paid developments ourselves as soon as possible?
"Believe the update for amounts above 1000 had raised its head earlier and there are some fix information from July probably an issue for you with the gateway version being used. The update was such a success even the bbc commented on the negative impact to business "
Well, it seemed to be mentioned as new since 1 August on the Protx forums so maybe some Protx servers switched earlier and some later.
Libraries on the Web
Conference papers of Association of Librarians and Information Professionals in the Social Sciences
Librarians against plagiarism: how Imperial College London is using PRS and active learning to combat the cut and paste generation. Ruth Harrison and Julia Garthwaite.
The practicalities of copyright in the online age. Helen Bartlett Copyright Manager, HERON.
Copyright and data licensing, does electronic differ from print?' Richard Ebdon, Copyright Officer, The British Library.
Rights and responsibilities: Copyright and digital images. Grant Young, Technical Research Officer, TASI - Technical Advisory Service for Images
IPR and multimedia in institutional repositories: lessons from the MIDESS project (Management of Images in a Distributed Environment with Shared Services) Lesley Pitman Librarian and Director of Information Services, UCL SSEES Library.
Irish Studies Online - a digital library of core resources for Irish studies. QUB. Norma Menabney, Queen University, Belfast.
I've not finished reading them myself yet.
Has an easy-to-use UK map survived the "if it isn't broken yet, keep adding features" fad?
So, besides Streetmap for now, is there an easy-to-use UK web map with contours?
hair transplant commented:
"to blog in different languages, we need and on line translator, I think"
in reply to my old question about multilingual blogging.
Like many multilanguage speakers I know, I don't hold out much hope of machine translation becoming useful any time soon. Systems like systrans can be useful for getting the basics of a message when you're totally stuck, but babelfishisms are a well-known danger (or comedy source, depending on what you're doing with it).
So we're left with the question of how to blog in multiple languages without annoying the hell out of everyone. Do aggregators support content-negotiation? Should we use bertilow-style consecutive translation? Or simply run multiple blogs?
I'm still only blogging in English. I probably should experiment. What's the state of this art now?
Comments are moderated (damn spammers) but almost anything sensible gets approved (albeit eventually). If you give a web address, I'll link it. I won't publish your email address unless you ask me to, but I'll email you a link when the comment is posted, or the reason why it's not posted.