Comments on: Setting up our own (small size) personal cloud infrastructure. Part #1, components

By Monday, November 3, 2014 Projects, Resources, Sciences & Technology, X-Posts Tags: 0057, B, C, Clouds, Data, Datacenter, Infrastructure, Sharing, Software, Tools Permalink 0

Following Patrick’s post about our different options for choosing a “Cloud” software and the one that we finally made by choosing ownCloud. Here are a few related comments that develop our point of view and technical choices.

ArkOs, Openstack and RiakCS all take the hand over an entire server/system/computer, offering a kind of embedded linux system within a human-friendly interface, the kind of mechanism one can find on ready-to-use NAS (Network Attached Storage) hardware.

Basically, it transforms any regular computer into a NAS device. One of the key points about the structure we are trying to setup is to be able to host anything we would like/need or may appear interesting to probe. That includes our own website(s), web services in order to feed projects with data and any kind of applications that may be useful to try and develop within the frame of this research.

We do need therefore to keep the research server as generic as possible by using a normal linux distribution, which we can then enhance by any set of additional services. While ArkOS, Openstack and RiakCS are of course interesting projects, at some point, it may become already too specific for our goals.

 

iic_server_05m

 

Owncloud appears to be a simple web site structure dedicated to file sharing. As mentioned in my previous post, Owncloud proposes a set of APIs that allow the access to Owncloud features while being able to develop our own applications. Thus, these applications can rely on Owncloud while being hosted on a heterogeneous set of devices, network connected.

Setting up our own (small size) personal cloud infrastructure. Part #1, components

By Sunday, November 2, 2014 Projects, Resources, Sciences & Technology, X-Posts Tags: 0056, A, C, Clouds, Data, Datacenter, Infrastructure, Software, Tools Permalink 0

On the way to the development of different artifacts for the design research (Inhabiting & Interfacing the Cloud(s)), we’ll need to work with our own “personal cloud”. The first obvious reason is that we’ll need a common personal platform to exchange research documents, thoughts and work together between the different (geographically distributed) partners involved in the project. We are thus our own first case study. The second one is to exemplify the key components about how a small data center / cloud infrastructure might be assembled today and learn from it.

But then, more importantly, this will become necessary for two other main objectives: first one is that we would like to give access to “cloud” tools to the design, architecture and makers communities, so that they can start play and transform a dedicated infrastructure (and this won’t of course be possible with all type of systems); second one will possibly be for the “hands on” and “prototyping” parts of our research, for which we’ll need an accessible cloud based architecture to modify or customize (this includes both the software and hardware), around which some open developments, networked objects, new interfaces, apps, etc. could be developed.

We are therefore looking for a (personal) cloud technology upon which we could keep (personal) control and this has revealed not so easy to find. Cloud computing is usually sold as a service that “users” are just supposed to… use, rather than a tool to customize or with which to play.

We would like of course to decide what to do with the data we’ll provide and produce, we’ll need also to keep access to the code so to develop the functionalities that will certainly be desired later. Additionally, we would like to provide some sort of API/libraries dedicated to specific users (designers, makers, researchers, scientists, etc.) and finally, it would be nice too if the selected system could provide computing functionalities, as we don’t want to focus only on data storage and sharing functions.

 

Our “shopping list” for technology could therefore look like this:

- On the software side, we’ll obviously need an open source “cloud” system, that runs on an open OS as well, to which we’ll be able to contribute and upon which we’ll develop our own extensions and projects.

- If a “hobbyists/makers/designers/…” community already exists around this technology, this would be a great plus.

- We’ll need a small and scalable system architecture. Almost “Torrent” like, which would allow us to keep a very decentralized architecture.

- Even if small, our hardware and infrastructure must be exemplary of what a “data center” is, today, so to help understand it. It happens that the same concepts are present at different scales and that, very simply said, large data centers just look like scaled, more complex/technical and improved small ones… Our data center should use therefore existing norms (physical: the “U“, rack and servers’ cabinet units), air (or water) circulation to cool the servers, temperature monitoring, hardware redundancy (raid disks, internet access, energy plugs).

 

We are scouting on these questions of software and hardware for some time now and at the time I’m writing this post, our I&IC blog is hosted on our own system in our own little “data center” at the EPFL-ECAL Lab…

 

IMG_3863_m

iic_server_05m

 

We’ve been through different technologies that were interesting, yet for different reasons, we didn’t really find the ideal system though. I can mention here a few of the systems we’ve checked more deeply (Christian Babski will certainly comment this post in more details later so to help better understand our choice as he, the scientist, finally made it after several discussions):

 

bittorrent_sync

 

We’ve seen some technologies that are based on a Torrent (or similar) architecture, like BitTorrent Sync (that kill the need for data centers) or a crowdfunded one like Space Monkeys. Both answer to our interest in highly decentralized system architectures and infrastructure. Yet they don’t offer development capabilities. For the same reason, we’re not taking into account open NAS ones like FreeNAS because they don’t offer processing functions and focus on passive data storage.

 

arkos

 

We should also mention one more time arkOS, that is both a very light and interesting alternative hardware/software solution, but that happens to be probably too light for our goals (too low computing and storage capacities on Raspberry Pis).

 

openstack

 

We’ve then continued with the evaluation of software that were used by corporations or large cloud solutions and that became open recently, for some parts and for some of then. Openstack seems to be the name that pops up more often. It offers many of the functionalities we were looking for, but yet doesn’t have an API and is certainly too low level for the design community, heavy to manage either. Same observation for Riak CS (scalability, but only file storage) that is linked with Amazon S3.

Of course, we shouldn’t forget to mention here that even big proprietary solutions (like Dropbox, Google Drive, Amazon S3 or EC2, etc. or even Facebook, Twitter and the likes that are typical cloud based services) offer APIs for developers. But this is under the same “user agreement” that is used for their other “free” services (subject to change, but where the “problem” is mentioned in the agreement: you’re a user and will remain a user). These options are not relevant in our context.

 

owncloud

 

We’ve finally almost found what we were looking for with OwnCloud, which is an open source cloud software that can run on a Linux OS, not too low level, with a community of developers and hobbyists, APIs, the standard cloud functionalities already well developed with desktop, iOS and Android clients, some apps. They recently added a server scalability that opens toward highly decentralized and scalable system architecture, which makes it compatible with our Preliminary intentions.

The only minus points would be that unlike “torrent like” architectures, it maintains a centralized management of data and doesn’t offer distributed computing. This means that once you’ve set up your own personal cloud, you’ll become the person who will manage the “agreements” with the “users”. This later point could possibly be addressed differently though, now that scalability and decentralization has been added. Regarding computing, as the system is installed on a computer server, with an OS, we maintain server side computing capacities, if necessary.

This will become the technical base of some of our future developments.

-

Please see also the following related posts:

Setting up our own (small size) personal cloud infrastructure. Part #2, components

Setting up our own (small size) personal cloud infrastructure. Part #3, components

 

Graffiti in the Data Center

By Friday, October 31, 2014 Society Tags: 0054, Datacenter, Infrastructure, Sociology Permalink 0
GraffitiDC

Since 2005, Google is interested in shipping data centers installed in 40” standard international containers. However, they also worked on the setup of the whole data center building  composed of these containers. This video walks us through one of these actual buildings. The screenshot above is a rare documentation of an exception addressing one of the eeriest aspects of data centers: the absence of any trace of human life or moreover social life, as depicted in Timo Arnall’s Time Machine. Graffiti is however a particularly redundant trace of human activity which reaches the most hostile environments, and in this particular sense it is interesting to consider two aspects.

First, in contemporary urban landscapes graffiti is often considered as a re-appropriation of public space. Therefore, in the case of the data center, which is obviously a corporate space, it seems to underline the socially problematic and blurry boundaries between public data and corporate data. Second, graffiti is a form of asemic writing, as well as undeciphered texts. As handwriting fades, data is carefully kept in redundancy, and as theorists and philosophers like Michel Thévoz or Jean Baudrillard underline, contemporary graffiti was born in the same years as the computer, the photocopier and modern replication technologies. It was (re)born through the paradigm shift from physical architectures to information architectures. It seems as if when internet grew out of Chatrooms and posted on the Wall, it lost the scribles.

–-

Via Youtube

Rack Mount Cases

By Friday, October 31, 2014 Sciences & Technology Tags: 0053, Cabinets, Hardware, Mobility Permalink 0
RackMountWeb

A rack mount cases is a transit case with provision for mounting rackmount equipment such as amplifiers, computers, displays, radios, encryption devices, and so forth. In many cases, the internal 19-inch rack is mounted to the transit case via shock absorbing mounts giving the rack sway space to attenuate shocks and bumps that might be seen during shipment and handling. One of the main standards in the design of these cases is MIL-STD-2073-1D – Standard Practice for Military Packaging. This can be linked to products of mobile data centers like Dell’s one for the military, which is based on a smaller scale than Googles 40” Shipping containers, for extra transportability. You can have an look at the product via Green (low Carbon) Data Center Blog.

–-

Source: http://en.wikipedia.org/wiki/Transit_case

From Bergcloud to ?

By Wednesday, October 29, 2014 Design, Projects, Resources Tags: 0052, Clouds, Hardware, Software Permalink 0

Even if unfortunately Bergcloud is a dead project now (and that it sadly brought down the design collective Berg too), I mention it here on our research blog as it probably has some connections with our own research project and what we might try to develop later as tools. The unsuccessful commercial approach of Bergcloud to connected objects should also be taken as a question toward this big “buzz” though.

 

bergcloud

 

One of our interest would be to develop a link (api) that could help designers tap into an (their own?) open cloud infrastructure, so to help them develop design artifacts that will take advantage of such third parties data/processing services. Providing such tools would certainly contribute in a large increase of alternative contributions that would go far beyond simple data/file storage or exchange…

You would possibly need then: an open, small scale yet scalable data center architecture (hardware and software), an open api in a programming language that would be accessible to designers and possibly a service or something similar to help play/share with (open) data.

Workshops that will be organized later in November with designer students at ECAL and HEAD, then later in January 2015 with architects at the EPFL, could help us fine tune the needs that such a API should try to cover.

Reblog > Internet machine

By Wednesday, October 29, 2014 Interaction Design, Resources, Sciences & Technology Tags: 0051, Conditioning, Data, Datacenter, Designers, Infrastructure, Internet, Networks, Research Permalink 1

Note: an interesting “documentary” project/resource for our project is this recent work by designer and researcher Timo Arnall. It was published last May on his website and on different blogs. The focus is obviously here on the data center as a (fascinating?) contemporary artifact, in which the search for technical efficiency, rationality, security, redundancy, clean air, modularity, etc. leads to a specific spatial aesthetic. It is this aesthetic that seems to become the “main character” for this movie that mixes techniques, even so the idea is to reveal/desacralize the “hidden materiality of our data”. This project is planned to be displayed as a multi-screen installation.

While I didn’t have the occasion to see the movie yet, we should keep it in mind and try to display it in the frame of our own research, in particular to the students that will take part to the different I&IC workshops.

 

Via elasticspace

—–

Internet machine is a multi-screen film about the invisible infrastructures of the internet. The film reveals the hidden materiality of our data by exploring some of the machines through which ‘the cloud’ is transmitted and transformed.

 

Film: 6 min 40 sec, digital 4K, 25fps, stereo.
Installation: Digital projection, 3 x 16:10 screens, each 4.85m x 2.8m.
Medium: Digital photography, photogrammetry and 3D animation.

 

Internet machine (showing now at Big Bang Data or watch the trailer) documents one of the largest, most secure and ‘fault-tolerant’ data-centres in the world, run by Telefonica in Alcalá, Spain. The film explores these hidden architectures with a wide, slowly moving camera. The subtle changes in perspective encourage contemplative reflection on the spaces where internet data and connectivity are being managed.

In this film I wanted to look beyond the childish myth of ‘the cloud’, to investigate what the infrastructures of the internet actually look like. It felt important to be able to see and hear the energy that goes into powering these machines, and the associated systems for securing, cooling and maintaining them.

 

InternetMachine14-web

 

What we find, after being led through layers of identification and security far higher than any airport, are deafeningly noisy rooms cocooning racks of servers and routers. In these spaces you are buffeted by hot and cold air that blusters through everything.

 

InternetMachine09-web

 

Server rooms are kept cool through quiet, airy ‘plenary’ corridors that divide the overall space. There are fibre optic connections routed through multiple, redundant, paths across the building. In the labyrinthine corridors of the basement, these cables connect to the wider internet through holes in rough concrete walls.

 

InternetMachine16-web

 

Power is supplied not only through the mains, but backed up with warm caverns of lead batteries, managed by gently buzzing cabinets of relays and switches.

 

InternetMachine10-web

 

These are backed up in turn by rows of yellow generators, supplied by diesel storage tanks and contracts with fuel supply companies so that the data centre can run indefinitely until power returns.

 

InternetMachine03-web

 

The outside of the building is a facade of enormous stainless steel water tanks, containing tens of thousands of litres of cool water, sitting there in case of fire.

 

InternetMachine11-web

 

And up on the roof, to the sound of birdsong, is a football-pitch sized array of shiny aluminium ‘chillers’ that filter and cool the air going into the building.

 

InternetMachine15-web

 

In experiencing these machines at work, we start to understand that the internet is not a weightless, immaterial, invisible cloud, and instead to appreciate it as a very distinct physical, architectural and material system.

 

 

Production

Internet machine shoot

 

This was a particularly exciting project, a chance for an ambitious and experimental location shoot in a complex environment. Telefónica were particularly accommodating and allowed unprecedented access to shoot across the entire building, not just in the ‘spectacular’ server rooms. Thirty two locations were shot inside the data centre over the course of two days, followed by five weeks of post-production.

 

Internet-Machine-production-04

 

I had to invent some new production methods to create a three-screen installation, based on some techniques I developed over ten years ago. The film was shot using both video and stills, using a panoramic head and a Canon 5D mkIII. The video was shot using the Magic Lantern RAW module on the 5D, while the RAW stills were processed in Lightroom and stitched together using Photoshop and Hugin.

 

Internet-Machine-production-02

 

The footage was then converted into 3D scenes using camera calibration techniques, so that entirely new camera movements could be created with a virtual three-camera rig. The final multi-screen installation is played out in 4K projected across three screens.

There are more photos available at Flickr.

 

Internet machine is part of BIG BANG DATA, open from 9 May 2014 until 26 October 2014 at CCCB (Barcelona) and from February-May 2015 at Fundación Telefónica (Madrid).

Internet Machine is produced by Timo Arnall, Centre de Cultura Contemporània de Barcelona – CCCB, and Fundación Telefónica. Thanks to José Luis de Vicente, Olga Subiros, Cira Pérez and María Paula Baylac.