Difference between revisions of "Airplane"

From HSG Wiki
Jump to: navigation, search
(start)
 
(Document guests + host metrics)
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
A generously donated HP ProLiant DL380p Gen8.<br />
 
A generously donated HP ProLiant DL380p Gen8.<br />
  
−
==Hypervisor==
+
=Hypervisor=
  
−
===Proxmox VE===
+
==Proxmox VE==
 
A.k.a PVE, an open source type 2 hypervisor based on QEMU/KVM and LXC. [https://pve.proxmox.com/wiki/Main_Page wiki]<br />
 
A.k.a PVE, an open source type 2 hypervisor based on QEMU/KVM and LXC. [https://pve.proxmox.com/wiki/Main_Page wiki]<br />
 
Apparently takes ~36GB to install, while using ~1.7GB ¯\_(ツ)_/¯.<br />
 
Apparently takes ~36GB to install, while using ~1.7GB ¯\_(ツ)_/¯.<br />
  
−
====Installation====
+
===Installation===
 
# put their ISO on a flash drive
 
# put their ISO on a flash drive
−
# boot from the flash drive
+
#* Alternatively, load an URL to the .iso in iLO's virtual media's CD/DVD-ROM URL field and skip step 3.
 +
#** The target URL may have to be on the LAN.
 +
# boot from the selected media
 
# get lucky, else try again
 
# get lucky, else try again
 
#* if it ends in <code>can't find cdrom</code>
 
#* if it ends in <code>can't find cdrom</code>
−
#** explicitly booting from USB (F11) may help
+
#** explicitly booting from USB (<code>F11</code>) may help
−
#*** Issue, keyboard doesn't work unless it's a cold boot (no F11 on attached keyboard when rebooting)
+
#*** Issue, keyboard doesn't work unless it's a cold boot (no <code>F11</code> on attached keyboard when rebooting)
 
#* the <code>[firmware bug] ... Corrupted Hw-PMU ... (MSR 38d is 330)</code> message is a negligible warning.[https://forum.proxmox.com/threads/firmware-bug-the-bios-has-corrupted-hw-pmu-resources.12469/]
 
#* the <code>[firmware bug] ... Corrupted Hw-PMU ... (MSR 38d is 330)</code> message is a negligible warning.[https://forum.proxmox.com/threads/firmware-bug-the-bios-has-corrupted-hw-pmu-resources.12469/]
 
# continue the installation process
 
# continue the installation process
−
#* the [[Airplane#Logical Drive 01|RAID1]] is shown as a 136GB drive
+
#* If both SAS drives are visible, make a ZFS RAID1 from them.
 
#* consider using the hostname <code>airplane</code>
 
#* consider using the hostname <code>airplane</code>
 
# upon completion,
 
# upon completion,
 
#* the server boots into PVE even if its boot priority is set to USB first
 
#* the server boots into PVE even if its boot priority is set to USB first
 +
#** It won't do this the next time, change boot order or remove the instalation media if you want to boot from hdd next time.
 
#* a web based interface is reachable via <code>https://airplane:8006</code>
 
#* a web based interface is reachable via <code>https://airplane:8006</code>
 
#* it is reachable via SSH
 
#* it is reachable via SSH
 
#** Address this before exposing this device to The Internet!
 
#** Address this before exposing this device to The Internet!
  
−
====Updating====
+
===Updating===
 
Without a subscription, apt tries to get the enterprise packages anyway.<br />
 
Without a subscription, apt tries to get the enterprise packages anyway.<br />
  
−
==Storage==
+
Commenting out the single line in <code>/etc/apt/sources.list.d/pve-enterprise.list</code><br />
 +
and running
 +
<nowiki>echo "deb http://download.proxmox.com/debian/pve stretch pve-no-subscription" > /etc/apt/sources.list.d/pve-install-repo.list</nowiki>
 +
should fix that.<br />
 +
If not already present, it may also be necessary to run this,
 +
<nowiki>wget http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-5.x.gpg</nowiki>
 +
[https://forum.proxmox.com/threads/error-401-updating-the-system.34567/#post-174403]
 +
 
 +
=Storage=
 
This server takes 2.5" SAS and SATA drives in G8 (8th generation) trays, and breaks compatibility with G7 and earlier trays.[https://serverfault.com/questions/401976/are-new-hp-gen8-servers-compatible-with-g7-drives]<br />
 
This server takes 2.5" SAS and SATA drives in G8 (8th generation) trays, and breaks compatibility with G7 and earlier trays.[https://serverfault.com/questions/401976/are-new-hp-gen8-servers-compatible-with-g7-drives]<br />
 
it has 2 "boxes" with 4 bays each, currently bays 1-4 are occupied, bays 5-8 are missing trays, $7.5-8 seems like a typical price (missing chip/LED?). (3D print? [https://www.thingiverse.com/thing:2665391])<br />
 
it has 2 "boxes" with 4 bays each, currently bays 1-4 are occupied, bays 5-8 are missing trays, $7.5-8 seems like a typical price (missing chip/LED?). (3D print? [https://www.thingiverse.com/thing:2665391])<br />
 
(microserver gen8 tray != proliant gen8 tray)<br />
 
(microserver gen8 tray != proliant gen8 tray)<br />
  
−
===SAS/SATA controller===
+
==ZFS RAID1==
−
On-board smart array SAS/SATA controller.<br />
+
Consists of LD 1 and LD 2, created by proxmox at installation.<br />
−
Commenting out the single line in <code>/etc/apt/sources.list.d/pve-enterprise.list</code><br />
+
 
−
and running
+
==BTRFS JBOD==
−
echo "deb http://download.proxmox.com/debian/pve stretch pve-no-subscription" > /etc/apt/sources.list.d/pve-install-repo.list
+
Managed in proxmox, created with <code>mkfs.btrfs -d single /dev/sdc /dev/sdd</code>.<br />
−
should fix that.<br />
+
 
−
It may also be necessary to run this,
+
==SAS/SATA controller==
−
wget http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-5.x.gpg
+
On-board Smart Array P420i SAS/SATA controller.<br />
−
[https://forum.proxmox.com/threads/error-401-updating-the-system.34567/#post-174403]
+
 
 +
===Box 1===
  
−
====Logical Drive 01====
+
====Bay 1====
−
2x HP SAS 146GB hard drives (15K rpm) in RAID1, bays 1 and 2<br />
+
HP SAS 146GB hard drives (15K rpm)<br />
 +
Logical Drive 1<br />
 +
<code>/dev/sda</code><br />
  
−
Controlled via the smart array [[Airplane#SAS/SATA controller|SAS/SATA controller]]<br />
+
====Bay 2====
 +
HP SAS 146GB hard drives (15K rpm)<br />
 +
Logical Drive 2<br />
 +
<code>/dev/sdb</code><br />
  
 
====Bay 3====
 
====Bay 3====
−
WD SATA 249GB hard drive
+
WD SATA 249GB hard drive<br />
 +
Logical Drive 3<br />
 +
<code>/dev/sdc</code><br />
  
−
====Logical Drive 02====
+
====Bay 4====
−
1x WD SATA 160GB hard drive, bay 4<br />
+
WD SATA 160GB hard drive<br />
−
This logical drive is of type RAID0, so is missing (a) drive(s), its data may presumably be lost, deleting the "drive" may fix the cache [[Airplane#Errors|error]].
+
Logical Drive 4<br />
 +
<code>/dev/sdd</code><br />
  
−
====Errors====
+
===Errors===
−
* [[Airplane#Logical Drive 02|Logical Drive 02]] thinks it's a RAID0, Failed
 
 
* Cache Module Status, Failed
 
* Cache Module Status, Failed
−
** [[Airplane#Logical Drive 02|Logical Drive 02's]] error may be inhibiting caching as a safety precaution?
+
** Persists after fixing (previous) logical drive 02's error, [https://community.hpe.com/t5/ProLiant-Servers-ML-DL-SL/Cache-Module-Status-failed/td-p/6925472 may be a capacitor issue].
  
 +
===Array Configuration Utility===
 +
The configuration utility is reachable at boot (<code>F5</code>)<br />
 +
There is also a commandline utility equivalent for linux known as <code>ssacli</code>, previously known as <code>hpacucli</code>, setup instructions can be found [https://serverfault.com/a/924919 here].<br />
  
−
The configuration utility is reachable at boot (unknown F key, read screen output)<br />
+
Passing all drives directly to the hypervisor (HBA) does not seem supported,<br />
 +
the simplest way to achieve this effect seems to be to make Logical Drives consisting of 1 Physical Drive each in the array controller.<br />
 +
It is currently unknown if this results in direct passthrough (is the [[Airplane#ZFS RAID1|RAID]] portable?).<br />
  
−
Passing all drives directly to the hypervisor and letting it handle them may be preferable (maybe not with ESXi)<br />
+
=Fans=
−
 
 
−
==Fans==
 
 
This server was named for the sound it makes at liftoff.<br />
 
This server was named for the sound it makes at liftoff.<br />
 
It can run quietly in normal usage yet currently does not, this should be addressed...<br />
 
It can run quietly in normal usage yet currently does not, this should be addressed...<br />
−
[https://wiki.archlinux.org/index.php/fan_speed_control#Fancontrol_(lm-sensors) fancontrol] may be of use but be cautions (you fry it, you buy it).<br />
+
[https://wiki.archlinux.org/index.php/fan_speed_control#Fancontrol_(lm-sensors) Fancontrol] cannot find any PWM controls.<br />
 +
 
 +
=VM setup=
 +
==Web interface==
 +
# Upload an ISO via <code>Datacenter > airplane > local</code> where you can click the <code>upload</code> button and pick a file on your computer.
 +
#* If a suitable ISO is already on the system you can skip this step.
 +
# Click <code>Create VM</code> at the top right and go through the wizard, select your ISO on the OS tab and fill in specs as needed (try to be conservative).
 +
# After clicking <code>Finish</code>, your VM should appear to the left under <code>Datacenter > airplane</code>, clicking it will bring you to a summary, and allow you to start it via the button at the top.
 +
# The console tab emulates a display connected to the machine, where you can set up your OS.
 +
 
 +
=LXC setup=
 +
==Web interface==
 +
# This step is similar to the first of [[Airplane#VM setup|VM setup]], except you click the <code>template</code> button and select a template for a system or turnkey container to download.
 +
#* for more templates, see [https://pve.proxmox.com/wiki/Linux_Container#pct_container_images PVE's wiki about <code>pveam</code>]
 +
# This step is again similar to the second of [[Airplane#VM setup|VM setup]], with the added requirement of providing a password.
 +
#* By default it wants you to provide a static IP, switching this to DHCP may be advantageous.
 +
# See steps the third and fourth steps of [[Airplane#VM setup|VM setup]].
 +
 
 +
=Metrics=
 +
Metrics are exported to a [[TIG]] stack on [[Spook]] using a [[Telegraf]] agent running on [[Proxmox]].<br />
 +
Also sends additional metrics directly to InfluxDB as [https://pve.proxmox.com/wiki/External_Metric_Server such].<br />
 +
Note: Telegraf's IPMI input requires a permission change on <code>/dev/ipmi0</code>, <code>chmod 666</code> seems to work.<br />
 +
 
 +
=Guests=
 +
List of set up guest systems, if it's listed here, don't remove it.<br />
 +
 
 +
===Spook===
 +
Hosts a [[TIG]] stack.<br />
 +
Once this system becomes "stable" (maybe after the fan noise is fixed), it is intended to also become the [[MQTT]] broker.<br />
 +
 
  
 
[[Category:Hackerspace]]
 
[[Category:Hackerspace]]

Latest revision as of 21:22, 8 January 2019

A generously donated HP ProLiant DL380p Gen8.

Hypervisor[edit]

Proxmox VE[edit]

A.k.a PVE, an open source type 2 hypervisor based on QEMU/KVM and LXC. wiki
Apparently takes ~36GB to install, while using ~1.7GB ¯\_(ツ)_/¯.

Installation[edit]

  1. put their ISO on a flash drive
    • Alternatively, load an URL to the .iso in iLO's virtual media's CD/DVD-ROM URL field and skip step 3.
      • The target URL may have to be on the LAN.
  2. boot from the selected media
  3. get lucky, else try again
    • if it ends in can't find cdrom
      • explicitly booting from USB (F11) may help
        • Issue, keyboard doesn't work unless it's a cold boot (no F11 on attached keyboard when rebooting)
    • the [firmware bug] ... Corrupted Hw-PMU ... (MSR 38d is 330) message is a negligible warning.[1]
  4. continue the installation process
    • If both SAS drives are visible, make a ZFS RAID1 from them.
    • consider using the hostname airplane
  5. upon completion,
    • the server boots into PVE even if its boot priority is set to USB first
      • It won't do this the next time, change boot order or remove the instalation media if you want to boot from hdd next time.
    • a web based interface is reachable via https://airplane:8006
    • it is reachable via SSH
      • Address this before exposing this device to The Internet!

Updating[edit]

Without a subscription, apt tries to get the enterprise packages anyway.

Commenting out the single line in /etc/apt/sources.list.d/pve-enterprise.list
and running 

echo "deb http://download.proxmox.com/debian/pve stretch pve-no-subscription" > /etc/apt/sources.list.d/pve-install-repo.list

should fix that.
If not already present, it may also be necessary to run this, 

wget http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-5.x.gpg

[2]

Storage[edit]

This server takes 2.5" SAS and SATA drives in G8 (8th generation) trays, and breaks compatibility with G7 and earlier trays.[3]
it has 2 "boxes" with 4 bays each, currently bays 1-4 are occupied, bays 5-8 are missing trays, $7.5-8 seems like a typical price (missing chip/LED?). (3D print? [4])
(microserver gen8 tray != proliant gen8 tray)

ZFS RAID1[edit]

Consists of LD 1 and LD 2, created by proxmox at installation.

BTRFS JBOD[edit]

Managed in proxmox, created with mkfs.btrfs -d single /dev/sdc /dev/sdd.

SAS/SATA controller[edit]

On-board Smart Array P420i SAS/SATA controller.

Box 1[edit]

Bay 1[edit]

HP SAS 146GB hard drives (15K rpm)
Logical Drive 1
/dev/sda

Bay 2[edit]

HP SAS 146GB hard drives (15K rpm)
Logical Drive 2
/dev/sdb

Bay 3[edit]

WD SATA 249GB hard drive
Logical Drive 3
/dev/sdc

Bay 4[edit]

WD SATA 160GB hard drive
Logical Drive 4
/dev/sdd

Errors[edit]

Array Configuration Utility[edit]

The configuration utility is reachable at boot (F5)
There is also a commandline utility equivalent for linux known as ssacli, previously known as hpacucli, setup instructions can be found here.

Passing all drives directly to the hypervisor (HBA) does not seem supported,
the simplest way to achieve this effect seems to be to make Logical Drives consisting of 1 Physical Drive each in the array controller.
It is currently unknown if this results in direct passthrough (is the RAID portable?).

Fans[edit]

This server was named for the sound it makes at liftoff.
It can run quietly in normal usage yet currently does not, this should be addressed...
Fancontrol cannot find any PWM controls.

VM setup[edit]

Web interface[edit]

  1. Upload an ISO via Datacenter > airplane > local where you can click the upload button and pick a file on your computer.
    • If a suitable ISO is already on the system you can skip this step.
  2. Click Create VM at the top right and go through the wizard, select your ISO on the OS tab and fill in specs as needed (try to be conservative).
  3. After clicking Finish, your VM should appear to the left under Datacenter > airplane, clicking it will bring you to a summary, and allow you to start it via the button at the top.
  4. The console tab emulates a display connected to the machine, where you can set up your OS.

LXC setup[edit]

Web interface[edit]

  1. This step is similar to the first of VM setup, except you click the template button and select a template for a system or turnkey container to download.
  2. This step is again similar to the second of VM setup, with the added requirement of providing a password.
    • By default it wants you to provide a static IP, switching this to DHCP may be advantageous.
  3. See steps the third and fourth steps of VM setup.

Metrics[edit]

Metrics are exported to a TIG stack on Spook using a Telegraf agent running on Proxmox.
Also sends additional metrics directly to InfluxDB as such.
Note: Telegraf's IPMI input requires a permission change on /dev/ipmi0, chmod 666 seems to work.

Guests[edit]

List of set up guest systems, if it's listed here, don't remove it.

Spook[edit]

Hosts a TIG stack.
Once this system becomes "stable" (maybe after the fan noise is fixed), it is intended to also become the MQTT broker.

Retrieved from "https://wiki.hackerspace.gent/index.php?title=Airplane&oldid=7469"