In a previous post, I described an experimental method of mounting S3 as a virtual file system within a cloud instance. I’m still in the process of doing spring cleaning… although fall is basically here… but anyway cleanliness is generally overrated until it comes to the idea of getting web files organized properly in the cloud. So before I take a shower this morning I think I’ll finish moving some static content into content distribution networked storage bit buckets.

#!/bin/bash
# Asher Bond 2010
# http://www.asherbond.com/blog/2010/09/23/sizeup-sh-for-cleaning-house-i-mean-cloud/
# sizeup.sh [dir]
# sizes up the present working directory or some other directory
# by summarizing the directories inside. I use this script to
# make sure my cloud compute instances are storing files properly
# in walrus and s3 filesystems or google storage instead of cluttering
# the compute instance's internal file systems

if [ "$#" -gt 0 ] ; then
        cd $1;
fi

find . -maxdepth 1 -type d -exec du -hs {} \;

A Content Distribution Network is often more efficient than one point of delivery such as a single or centralized web server / VPS.

Please note this is risky business down here…
Here’s how a thrill seeker could try to move a whole bunch of stuff into a cdn real fast:

sizeup.sh
# oh wow what’s up here? that one is like fulla mp3s and videos
s3-mount.sh cdn.somedomain.com /mnt/cdn.somedomain.com
mv /var/some-directory/some-big-podcast-2010* /mnt/cdn.somedomain.com
echo “hey web master dude I just moved all ur files into the cdn, so update ur links.” | wall
# oh no I’m still the web master…
tar -cvf backup-in-case-my-links-dont-get-updated.tar /var/some-web-site
gzip *.tar
umount fuse
s3-mount.sh backups.somedomain.com /mnt/backups.somedomain.com
mv backup-in-case-my-links-dont-get-updated.tar.gz /mnt/backups.somedomain.com
cd /var/some-web-site
find . -type f -exec perl -p -i -e ‘s/some-old-links/some-new-cdn-links/g’ {} \;
# whoa i hope that worked… LOL!

In the future, computers will be more and more portable and the networks that connect computers together will be much faster. Data will be centralized for the purposes of logical integrity. This requires that networks be made fast, reliable, and distributed. Computing in the cloud could not come at a better time than now. Our now? IPV6 is our now and everybody’s now in everyone’s world, connected by our smart phones, laptops, desktops, and servers. E-commerce is going international and fully mobile. Hardware vendors are building systems smaller and faster than ever before. Open source software distribution models are transforming the commercial software model into a collective library… an understanding of technology at the programmatic level. People are freed and empowered to relate to each other by these advances. The commercial system now has a reduced load average and can focus on other processes, like buying, selling, and advertising online using mobile phones. Collective global positioning and mobile innovations have brought knowledge to freedom on foot. Supercomputing is here and we are distributing it to the world, empowering humans to spread ideas like nectar. Modularity, flexibility, convergence, and resilience are emerging in today’s software. What do you expect? – Asher Bond