welcome   Click to listen highlighted text! welcome Powered By GSpeech

budget.node.-.examples

while we went warm with theoretical package dense of up to 8x nodes in 1he server case, the reality is comming by and tell us, what is really up to do. the given hardware is:

  • 1x cubieboard1 | 1 hdd 500 gb
  • 1x cubieboard3 | 1 hdd 2 tb
cubieboard 1 + 3
cubieboard
2x + 1x

the price tag is 90 + 50 = 140 € for the boards alone, without any harddrive nor router nor accessoires.  the money is allready spent to the development god of tiny devices with giant duties. it could handle 100 mbit/s to the outside world, or with some usb-ethernet-adapter possible the 1000 mbit/s from the bigger one, but on some point, the conversion/transversion from gigabit ethernet to 100 mbit/s must be done. the best way would be a board with dual-network-connector, but its not available yet.

 

 


  • 1x cubieboard3
  • 1x 2 core
  • 1x 1 gb ram
  • 1x 100 mbit/s / 1000 mbit/s nic
  • 1x 500 gb / 2 tb hdd
  • replication factor 1
  • price: 1 board + 1 hdd = 90 € + 90 € = 180 €
node config 2X
cubieboard 3
node config 2X

thats the current state. as the network extension cable is not working well, as we tried to wire the two leds for connection and traffic to the outside world. so the 100 mbit/s / 1000 mbit/s version is ready to gamble, but just with a replication factor of 1. thats poor, that less then raid 1 to compare it a bit. the harddrive should never fail or the backup for 500 gb is able to sync. within 24 hours on dsl-speed, at home. two fans should be ok for that amount of power consumption.

 

 

 

 


  • 1x cubieboard1 + 1x cubieboard3
  • 1x 1 core + 1x 2 core = 3 core
  • 1x 2gb ram + 1x 1 gb ram = 3 gb ram
  • 1x 100 mbit/s / 1000 mbit/s + 1x 100 mbit/s nic = 200 mbit/s
  • 1x 2 tb hdd + 1x 500 gb hdd = 500 + 1500 gb hdd
  • replication factor 2
  • price: 2 board + 2 hdd = 90 €  + 50 € + 2x 90 € = 320 €
node config 2X + 1X
cubieboard 1 + 3
node config 2X + 1X

here is the configuration with the available boards that i have for now.  line speed is like the same, as above, the logical bandwith would be 200 mbit/s if duplex is working without a loss. nothing needs to be shoped, all visible parts are on stack. screwing that into the case is another step, but then the system should run, even if on harddrive fails. a internal router is needed, 3 ports would be fine for that.  another way is, to connect the smaller load-balancer bord with a usb-ethernet-converter, but thats mystery from here on, as not every library is compiled on armhf. the power conversion needs to be doubled 4 ampere should be available for both boards.

 

 


  • 1x cubieboard1 + 2x cubieboard3
  • 1x 1 core + 2x 2 core = 5 core
  • 1x 1gb ram + 2x 2 gb ram = 5 gb ram
  • 1x 100 mbit/s + 2x 100 mbit/s / 1000 mbit/s nic
    = 300 mbit/s + 1800 mbit/s
  • 3x 2 tb hdd
  • replication factor 3
  • price: 3 board + 2 hdd = 2x 90 € + 50 € + 3x 90 € = 500 €
node config 2 x 2X + 1X
cubieboard 1 + 3
node config 2 x 2X + 1X

here is a version with a configuration of smooth compaction. logical bandwith would be 300 mbit/s or 2 x 1000 mbit/s + 100 mbit/s if the adapter is willing to switch it that way. another chance is a router, with a uplink, which handle that multiplex shifting. replication would be the minimum, as suggested. a internal router is needed, 4 or 5 ports would be fine for that. the power conversion needs more attention, estimated 6 ampere should be available therefore. three fans will blow the heat of the discs away, beside the little temperature fom the tiny boards.

 

 


its hard to find a conclusion on that. one the one hand 320 € are allready spent, with harddrives and that two boards.  the actual configuration costs around 180 €, but upgrading to the goal of replication factor 3, is another 180 € away. were in the middle of that price tag.

okay, the cheapest entry class server on dells online shop starts around 600 € for 2 core, 4 gb ram. for 745 € we could get 4 core and 8 gb ram, just to have more blast. to get replication 3, we need to spend another 400 € for additional sas-drives, makes it to 999 €, with the 2c / 4gb configuration. having more cpu and three drives, the price would be 1145 €. thats twice of our diy cloud-node and the example case to show up with.

we could have replication factor 3, but a quarter of cpu power available, compared to a pentium-class or xeon-based server with replication factor 1. having replication factor 3 there, with the bottleneck of a single board.


the independent boards could be configured via software as needed. thats called software-defined-network ( SDN ) nowadays. the single server board will be rougher as a developer board, but that would just be one node, with a unstriped disc-array. the cubie´s would present 3 nodes, for half the price, with a estimated four to six times slower arm v7 cpu. using the gpu with software will take another 1 or 2 year, from now on – if happen anywhere.

so, bringing it online the described upgrade way, would save the money to the point, when replication of 3 is urgently needed or when another chasis/case is ready to be deployed. then, the 2 + 3 drives would make the boom, the system is going to present for that issues.


serving web content is one case for that system. handling large files as no-sql engine, filebased with some glue, is the main goal within the logical network layer. synchronizing a database-cluster on one master and two slaves, would lower the bandwith for the need of replication. that will just scale between 2-5 or so. the distributed system would level that above 3 boards, in one server-chasis.

the network behaviour is not counted in this mind construct, a layer-3-switch would be very gentle to the paket-flow, but is outside the budget nor tested at all. switching the 5 – 8 available ports with 10 gbps would be fair enough, then the machine would have 1.5 gbyte/s inner bandwith, makes it half a iscsi device, targeted with single connect on 4 gbps.

mozilla.thunderbird.-.local.mail.size

als mail-client hab ich im laufe der jahre einige ausprobiert. früher outlook-express, was anfänglich nur einen mail-account verwalten konnte. dann das komplette outlook, das konnte dann verschiedene accounts verwalten. netscape mail, war auch nicht sooo hübsch, dann eine weile nur webmail und nun, hab ich mir eben seit einer ganzen weile schon, den mozilla thunderbird als portable version auf die platte getan.

der vorteil ist, man kann verschiedene versionen / konfigurationen nebeneinander betreiben und auch einfach mal ein backup machen. thunderbird ( und andere auch ) speichern mails und inhalte nicht nur auf dem mailserver, sondern ziehen sich kopien auf die lokale platte. bei mehreren accounts und einigen jahren archiv, kommen da schon ein paar megabyte zusammen. mir fiel es heute wieder auf, das dort ~800mb rumlagen. zu viel für meinen geschmack, über 1 gb sind schlecht zu durchforsten, falls ich mal was suchen will.

total commander
screeshot vom commander auf thunderbird verzeichnis

dies kann man mit der rechten maustaste auf ein verzeichnis im explorer oder finder,  unter eigenschaften kann man den verbrauch ablesen oder man markiert eine datei/verzeichnis mit der leertaste im totalcommander. wie man sieht, ist die größe des mülleimers (trash) im augenblick 2.640.510 kB. einige ordner möchte ich lokal puffern, der trash gehört aber nicht dazu.

eigenschaften von ordner
thunderbird – ordner eigenschaften

thunderbird ( version 24.4.0 – release channel ) pflegt zu jedem ordner seine eigenen einstellungen.  dort kann man, unter anderem, die synchronisation de/aktivieren. mit der rechten mausi-taste auf den ordner der wahl, dann ergibt sich das folgende fenster als context-menü quasi. yoh, den trash-ordner  wollte ich – wie gesagt – nicht offline speichern, da müllt einem nur die platte zu. ich hab diese synchronisation sehr selektiv eingestellt, manche ordner sind dort aktiv, andere nicht. hier im dritten reiter gibts diese auswahl, im ersten reiter dann diese:

eigenschaften von ordner
thunderbird – ordner eigenschaften

bei “Größe auf Datenträger:” stand vorhin die zwei komma irgendwas megabyte größe, der letzten gelöschten mails. jetzt befinden sich noch 3 mails im papierkorb, die gesamtgröße beträgt 3.5 kB, auf der platte werden 5.444 kB reserviert. löscht man nun die mail und klickt auf “Reparieren” wird der ordnerinhalt auch auf 0 kB gesetzt.

das kommt daher, dass thunderbird zwar die header-informationen speichert, beim anklicken der nachricht wird aber auch der inhalt/body geladen. hat man sich nun viele oder alle mails angeschaut, dann wächst dieser speicher eben (sinnlos) mit, wie in diesem fall. ich kann an dieser stelle drauf verzichten und freu mich lieber, das am ende der räumungsaktion nur noch ~260 MB auf der platte verbraucht werden. in ein paar monaten, spätestens in einem jahr, ist der ordner wieder auf 1 gb angewachsen, je nach mailaufkommen. der vorteil ist, das gelöschte mails auf dem server, dann immer noch lokal vorhanden sein könnten – eine letzte chance auf rettung kann dies sein, aber auch kompomitierender inhalt, aus längst vergessenen oder verdrängten konversationen :D

cubie.in.da.houze

im dezember wurde die olle dell-möhre ausgeschlachtet, nun hatte ich ein paar wochen/enden zeit, um die boards auf herz und nieren zu testen.

cubieboard mit einem kern, einem gigabyte ram und 100mbit/s würden als loadbalancer oder datanode vermutlich ausreichen, um ein paar sachen laufen lassen zu könne. die notwendigen dienste jedoch, brauchen aber in der mindest-konfiguration schon mehr als diese eine gigabyte, daher hab ich jetzt den cubietruck in das 19 zoll servergehäuse geflanscht.

cubietruck nearly installed
cubietruck inside the 19 inch standard server chasis, with 1 he

die pestflatte hat nun einen sata-anschluss, statt ultra-scsi und ist am steckverbinder mit den gehäuse verklebt. heißkleber hat sich in letzter zeit als durchaus brauchbar dargestellt, die mechanische belastung sollte aber dennoch nicht zu hoch sein. der taster zum einschalten und zwei leds zum netzteil bieten ein wenig einfluss und information von vorn.

da die spannung mit 12v aus dem netzteil kommt, das board aber 5v haben will, gibt es einen regler, der diese spannung mit  etwa 85% wirkungsgrad verkleinert. von lm 317 hab ich noch jede menge hier rumfliegen, aber in zeiten von längsreglern und schaltnetzteilen, nicht mehr ganz zeitgemäß. wie dem auch sei, der eine regler dient auch der versorgung der hdd, 2 A sollten da wohl in summe ausreichen. da im gehäuse eigentlich 4 kaskadierte lüfter ( also 8 insgesamt ) drehen und im netzteil auch solche lärmdrüsen eingebaut sind, hab ich 2 von denen ans ende versetzt, damit festplatte und board eine gewisse zwangslüftung erfahren. man weiß ja nie, wer unter einem so kocht ( welcher server direkt darunter hitze produziert ).

cubietruck on 19 inch
cubie from the rear side of the case

die zuleitung für die festplatte reicht knapp aus, dafür muss die netzwerkdose noch aus dem gehäuse geführt werden. die meisten rj-45 döschen haben eine anzeige, wenn elektrischer kontakt besteht und wenn traffic auf der leitung ist. diesen luxus bietet das cubieboard 3, doch die übliche cat5 oder cat6 leitung hat nur 4 doppel-päärchen, also 8 leitungen. für 2 leds brauchts aber auch nochmal 4 drähte, also muss eine zweite leitung her. als ersatzteilspender bietet sich das alte dual-sockel-mainboard an, dort sind zwei dosen aufgelötet. eine fürs netz und eine für recovery, wie dem auch sei, mit einem 160w mörder-lötkolben ist sie schnell getrennt vom board. eine heißluftpistole wäre auch schick gewesen, die hat aber vor kurzem ihren geist aufgegeben.

jedenfalls macht das netzwerk noch zicken, daher mal ich mal die nächste version schonmal auf. 2 cubietrucks + 2 x 4 tb hdd und 1 cubieboard + 1 x 4 tbb + 4/5 x switch gb …. to be continued….

landing.page.-.reworked

as i moved the landing page from plain html to a framework-based construct, i picked  up bootstrap for this issue. having jquery and css prepared, is half the way to a nice site.  i had plain javascript under construction as well, but fiddling from scratch with a handful of methods or classes, was to slow and in the end sucking, so i was happy to get a proper toolkit to work with.

landing page screenshot
landing page – section: scope

using version 2 of bootstrap did the trick over the past year, but as i noticed, the new version 3 was on the door.  new fonts, a bunch of icons and all for free. okay, some features are hard to find, but the result is a fast page and the dom-tree is ready in something like 800ms. using the social plugin from g+ makes the complete page a bit slow, but the scoring for search engine optimization (seo) will give the return.

furthermore, i implemented blueimp-gallery, font-awesome, bootswatch, piecemaker and jquery(mobile) as backbone for that libs. that is quite a lot, but as i wanted the page delivered by a servlet, the  structure and usability was more in the focus. beside that nice modules, the result and presentation is it worth. it took me a while to grow the inner organisation of the topics, the documentation and examples for each package is less then expected.

landing page screenshot
landing page – section: service/tech

the next step is, to use lesscss as condenser for the bloated javascript and stylesheet files, and shrinking them to the objects, that are realy needed for the site itself.  so i need to stick into node.js, npm and other superglue for server-side javascript. its not my cupcake, but it looks like i need to dive into that. running that on windows would need cygwin and my experiences with that leads me to work on pure linux.

landing page screenshot
landing page – section: service/tech

anyhow, the raw stuff is done, finetuning, colors, text and tiny changes will come the next time. but all-in-all, its just to have a presentable domain by hand.

teensy-3.1.-.256k.rom.-.64k.ram.-.arm.v4

what to to, when a dude smashes a harddrive from a mac? the new drive dosnt have a clue about the ingredients of the machine, the firmware/bios is like a sissy to not let the owner accessing the system. several ways to get into the mac osx, but over the years, the rules went harder and one should explain the story to the support at some point.

pcb
top view

 

apple puts a lot of energy into their politics, to seal the borders of that system and to save the mac from other influences, except theirselfs. icloud is nowadays a default package, the most services, like imessage, is backed up in the cloud. no way to use such devices without such id for that storage and control of puzzlepiece. the mac is locked until the code of consumption is entered or penalty is payed to apples money-machine via customer care. fucking with assemblercode, to tweak the bios with u/efi and all that shit, is not even a plan. the firmware to restart the system is hidden within some hashes from the mainboard and the password itself, reverse engineering was not my plan for today.

windows programmer
ready or not, depends on the configuration of buttons and lights. this picture indicates a working chip.

if the mac is hanging on the four number code, to get back to live, one could call the customer care to get a scbo file or try typing the nearly 9999 combinations by hand. others are lazy to do so, a small processor, based on arm with 64kb ram, can do the same, much more gently. the tiny pcb ( printed circuit board ) with a size of 35 x 17mm holds a usb-connector, the cpu with 256kb rom and a lot of connectors to participate with others.

the firmware tells its a v.1.03, flashed with the customized arduino ide   ( integrated desktop enviroment { development tool } ) within seconds on the freescale MK20DX256, but with some difficulties. arduino is not just a misswritten word, they could be seen as major pusher behind that uprising developer-board-scene. that teensy is a clon in some way, but in fact, arm or atmel had a product line over the past years, so one can find several configurations, sources and dealerprices, to get into the rush of diy, for a fistful flies. another eclipse plugin should be named, but i had so-so results on the download site, all files were incomplete,  i guess the traffic limit is taken, on the end of the month. the project should move to the next level then.

chip is ready to reboot
update cycle finished

as i hadnt much experience with avr/atmel chips at all, i decided, to put everything in a virtual box, on windows 7 for some reasons. but suprise suprise, the winavr-toolkit wants to lay inside the default folder ( ${winDrive/winavr } ) instead of being sorted with others in the standard folder for program files (x86). anyhow, the masterpiece of microsoft is, that the usb port and parameter is buffered/saved within the registry and due to other lacks of their understanding of software design, the programer wont flash the hex.code from the arduino software into the chip. excellent, couple of hours later i gave up on this, as i wanted to use eclipse for larger developments.

so, burning the working code to the chip was only possible, with a direct connect to the host-system.  some years ago, i played with a atmega ( think was …32 ) for a laser-tester, that one had started with basic-code. the arduino enviroment has its own syntax and file-extension.ino and with gnu, c/c++ software the architecture could be cascaded.

happy hour for the chip
upgrade sucessfullly, the new code is flashed to the soc

so, the chip is now ready to stupidly check each number, from 0000->9999 until the mac agrees to work again, when the correct passcode is clearly analyzed, by try& error. oh yeah, the soc could do much more then this, but the case was just there, to check the stuff with a goal to reach. some dozen projects and use-cases for that little machine is visible on the web: dj-controller, measurement boxes with lcd displays or game-engines are out there and many other ideas are just eyeblinks away.

bigger or larger projects will clearly need some more attention and notes to be realized, but with a sd-card-slot in dollar-distance, the bugs for a unique or genuine solution is definitly no high steak or far away. things can be made and grown with developer kits, analog or digital interfaces, up to the samplerate of a oscilloscope or analyzer by mean.

windows programmer
ready or not, depends on the usb connection and windows is a sissy in that area, after 15 years of existance, a unhappy programmer with no chance to pump new bytes into the chip.

kurzweil.-.zweiter.akt

after 4 weeks of drying the boards on my heating, i tryed to arrange a date with doogie, who helped on the phase one. not knowing, how the pitch-wheel and the floppy was connected, the synth was just closer to be doomed. but, after 6 weeks or 7 we arranged a double day, to slowly rebuild the kurzweil from its parts.

first test run
kurzweil loose connected

reassembling … how to start without a manual on that level. so … isolating the parts from the case and other connected parts, was in the focus here. the power supply was also not to trust, but with a headphone, we went sure, that no noise nor sine waves came from that part, beside the expected voltage.  it looked good for the moment, power was there, no cable was missing, the defect buttons were easy to change … so lets try it.

second test run
kurzweil loose connected

nope, nothing to see on the display, not lights on, no sound nor growling from a short cut were to hear, so we ripped all parts of the case and connected each cable to a part, that was numbered or signed in somehow. but … fail. some 2-3-4 ends of cables were without a mark. hm, …. we tried to start the engine, but one of the ribbon cables starts smoking, a sure sign of a short cut, that was not the right port for that cable.  we stoped and looked on each board, not seeing the difference, we have had to made. so we started again, and the cable went hot instantly – again.

kurzweil inside
upside down inside the kurzweil

the engineers on kurzweil showed us, where the fork is hanging around. pumping current over a tiny, 0.14 mm² of cable, couldnt be more worse. they used big cables for big wattages, but why did they did that different, on the engine board then? we looked closer after some minutes, or was it hours of cluesless talkings. we suprisingly found 2 connectors without a cable, so we switched something between the sampler, i think the scanner and the engineaudio analog or digitized, anyhow, could be the case. we stoped breathing when we switched on the power plug again … and … the display starts working. but, as i estimated on old-fashioned-hardware, each part of the system needs to be connected the way it should, so we couldnt check part by part, we had to stuff all together, with the hope, its allright then.

k2500 boards
k2500 display and scanner board

the five knobs or buttons on the input board, which were dead as distances holder, they were quickly replaced by some from the a-z/+-# keyboard. five pieces, let one cost 50 cent, made that accident happen, that we started to enjoy then later. the system came up, the boot-cycle went successfully done and the thing came to life. but having everything upside down, not knowing what buttons to push, we tried some intuitive haptics by accident, but after everyone of us three tried it, some preset with playable pattern suprised us thru the headphones and speakers with sound. the rest was routine to not mixup the three packs of screws and as we nearly closed the parts of the case, the kdfx board went back to my mind.

i had spare parts to drop in, with more power, more harmonic distortions and bigger rooms to simulate. but first, i wanted to get warm a bit with that tier, to check, if its worth the switch from software-based production to this kind of hardware. i would notice, it is it worth, but fiddling the menue was not funny, so the vast.programmer needs to be setup … next or the very next weekend … some day. that reopening, droping in, updating the firmware with a floppy drive, then stimulating to live once again will need a long way to get warm with that idea. one nice part is, that after that years, the software is still online on kurzweils server, that is a big plus in that whole story.

crunching on arm

as far as it looks like, iam not the only one, crunching on arm. dzone got a article from thailand, where some guys connected 22 cubieboards, runinning apache spark, for in-memory-computation.

but ive seen the bottleneck of a 100 mbits network connector on cubieboard pretty much earlier. so ive decided to buy the cubietruck, which includes a 1 gbits adapter, which brings theoretically 128 mbyte/s a sata drive for saturation.

a cluster from france was presented before on cubieboard.org with the same idea, to fit as many cubieboards within a standard rack server enclosure, as possible.

i calculated 8 cubieboards in a 1 he server rack as maximum amount of boards and hard discs last year. the price is then something like 1600-2000 € for the hardware. for that price tag, the memory of 16 gb ram and 16 tb of hdd is not that much, but the theroretical bandwith between the nodes ( within the cluster ) is something like 8 gbits :)

8 nodes
8 boards in a 1 he standard rack server

the picture above shows the former choosed boards from hardkernel, called odroid, but the price for that board is above the budget, so i sized it down to a dual-core, from the quad-core.

cloudAnbieter

da festplatten zwar immer günstiger und größer werden, die dateien und formate aber auch, bleibt das ewige spiel mit dembackup. wie möchte man eine 2-3-4 terabyte platte sichern? wie kann man 10 oder 20TB sinnvoll wegschaufeln? was tun, wenn das haus abgebrannt oder abgesoffen ist – was ist dann mit den eigenen daten. klar, man kann sie brennen oder auf usb kopieren, aber ab einer bestimmten größe macht das alles keinen spass mehr. nun hab ich heut beim g+ ein paar interessante beispiel für verfügbare clouds aufgeschnappt:

tarsnap fällt durch schlichte leistung und geringe kosten auf. 0.30us$/gb-monat für den space und 0.30us$/gb beim traffic scheinen günstig zu sein. der wesentliche punkt aber ist, dass nur veränderte daten hochgeladen werden, ein inkrementelles backup sinngemäß. aes-256 wird angeboten und kann, da linux-basierter-client, durch eigene maßnahmen ergänzt werden. als infrastruktur wird amazon ec2 verwendet, also nichts aufregendes.

crashplan sieht wesentlich ausgereifter und professioneller aus. die verschlüsselung beträgt 448-bit für die dateien und 128bit-aes für die verbindung. 4us$ für einen rechner, total-backup, klingt ok – eine konkrete größe vermisse ich aber dann doch. für firmenkunden werden auch pakete mit 200 oder 1000 und mehr angeboten. die infrasturktur ist nicht auf den ersten blick ausgeschildert.

der vollständigkeithalber wären noch die üblichen  verdächtigen zu nennen:

google-drive
apple-icloud
µ$-onedrive
strato-hidrive

klar kann man sich auch ein nas hinstellen, doch es geht ja um ein offsite-backup, also eine sicherungskopie ausserhalb der eigenen vier wände. sicher, im bunker, tief im rechenzentrum, dupliziert, verteilt, gespiegelt – einfach bombensicher *hust*. was die einzelnen betreiber jedoch im kleingedruckten für regeln und leistungen haben, muss man dann doch genauer evaluieren.

runInstall

so dat wordpress is nun wohl mal druff. ne handvoll plugins noch dazu … eventuell noch n theme .. dann reicht ditt für heut och schon gleich mal wieder …. zumindest für den augenblick.

  • wp super cache ( php / mod_rewrite caching – excellent )
  • wassup real time analytics ( see users online, without log-file )
  • xml sitemap & google news feed ( webmaster-tools will make seeking people happy )
  • math captcha ( stop bots with math to solve )

mal sehen, wie lang dat ding hier hält und wie ofts abschmiert….und wie oft der installer dann wohl läuft, bevor was anderes her muss. für die <code> fragmente muss ich auch noch ein gescheites plugin finden, ebenso ein hübsches, aber schlichtes template.

nun noch

  • wp easy backup ( complete backup of db, plugins and files )
  • broken link checker ( checks inherit links for availability )

installiert, dann klappts auch mit der restauration und der gebannten gefahr, das jemand in tote links rennt. einfach die namen bei der plugin-installieren eingabe reinkopieren und suchen lassen, dann sollte sich das korrekte teil schon finden lassen.

Click to listen highlighted text! Powered By GSpeech