Star Rating LEB/Ubuntu

Star Rating Table Structure:

  • Features ordered by Feature Category
  • Star Priority reflects what Star Rating a failure would block

Star calculation is simple:

  • Each Feature has a "Stars" Priority; it reflects which star count this feature blocks/supports.
    • rule: if a feature does not work, the accumulated star rating cannot have that star count as high as the priority of the feature
  • Overall Star Rating is the maximum star priority for all features for which all results were OK and for which and all lower priority features yield OK as well.

  • Feature Category Rating is the maximum star priority for all sub-features for which all results were OK and for which and all lower priority features yield OK as well.

Example: PandaBoard test results 11.06

Panda 11.06

Cycles/1105/BoardSupportStatus/enabled-3-star.png

  • Panda is a 3-star LEB/Ubuntu Board

    • to gain 4-star rating fix: Audio, Video Out - HDMI, Video Out - DVI, Networking, Bluetooth, USB

    • to gain 5-star rating fix: Graphics, Multimedia

Category Rating

Rating

Comment

Star Points / Blocks

Overall

Cycles/1105/BoardSupportStatus/enabled-3-star.png

Audio

Cycles/1105/BoardSupportStatus/enabled-3-star.png

ISSUES: audio on jacks

SUPPORTS: 3 / BLOCKS: 4

Video Out - HDMI

Cycles/1105/BoardSupportStatus/enabled-3-star.png

ISSUES: monitor hotplug, disable display

SUPPORTS: 3 / BLOCKS: 4

Video Out - DVI

Cycles/1105/BoardSupportStatus/enabled-3-star.png

ISSUES: all resolutions, monitor hotplug, disable display

SUPPORTS: 3 / BLOCKS: 4

Graphics

Cycles/1105/BoardSupportStatus/enabled-4-star.png

ISSUES: OpenGLES/EGL X11

SUPPORTS: 4 / BLOCKS: 5

Networking

Cycles/1105/BoardSupportStatus/enabled-3-star.png

ISSUES: Onboard Wireless Kernel

SUPPORTS: 3 / BLOCKS: 4

Bluetooth

Cycles/1105/BoardSupportStatus/enabled-0-star.png

ISSUES: Onboard Bluetooth Kernel

SUPPORTS: 4 / BLOCKS: 4

USB

Cycles/1105/BoardSupportStatus/enabled-2-star.png

ISSUES: performance

SUPPORTS: 2 - BLOCKS: 4

SD/MMC

Cycles/1105/BoardSupportStatus/enabled-3-star.png

ISSUES: -

SUPPORTS: 3 / BLOCKS: -

Serial

Cycles/1105/BoardSupportStatus/enabled-2-star.png

ISSUES: -

SUPPORTS: 2 / BLOCKS: -

Test Matrix

Category

Feature

Priority

state

notes

Audio

3

ISSUES: audio on jacks

HDMI audio

3

OK

audio on jacks

4

FAIL

no driver

Video Out - HDMI

4

ISSUES: monitor hotplug, disable display

all resolutions work

4

OK

monitor recognized

3

OK

native resolution used

3

OK

name of output should be changed bug 803424

xrandr lists resolutions

4

OK

xrandr changes resolution

4

OK

monitor hotplug

4

FAIL

recognized by x11 but no output is present

disabling output results in broken display

4

FAIL

bug 803494

Video Out - DVI

3

ISSUES: all resolutions, monitor hotplug, disable display

all resolutions

4

NEEDS WORK

resolution limited to 1280x960

monitor recognized

3

OK

native resolution used

3

OK

name of output should be changed bug 803424

xrandr lists resolutions

4

OK

name of output should be changed bug 803424

xrandr changes resolution

4

OK

monitor hotplug

4

FAIL

recognized by x11 but no output is present

disabling output results in broken display

4

FAIL

bug 803494

Graphics

4

ISSUES: OpenGLES/EGL X11

Framebuffer

2

OK

Framebuffer X11

3

OK

Framebuffer OpenGLES

4

OK

Framebuffer OpenGLES/EGL X11

5

NEEDS WORK

crashes badly when too many windows are open bug 803412

Networking

2

ISSUES: Onboard Wireless (ALL)

Onboard Ethernet Kernel

1

OK

Onboard Ethernet U-Boot

3

OK

Onboard Ethernet - ifconfig up/down

2

OK

Onboard Ethernet - dhclient

2

OK

Onboard Ethernet - network-manager

3

OK

Onboard Wireless Kernel

4

FAIL

driver disabled bug 785739 for omap2plus

Onboard Wireless - wireless-tools (scan, unprotective + dhclient)

4

FAIL

Onboard Wireless - wpasupplicant (WEP, WPA-PSK)

4

FAIL

Onboard Wireless - network-manager (clear, WEP, WPA-PSK, eth + wlan)

4

FAIL

Bluetooth

0

Bluetooth Kernel Driver

4

FAIL

BT address not set bug 789095

USB

2

Kernel/Controller

1

OK

Kernel/HID keyboard

2

OK

Kernel/HID mouse

2

OK

storage

2

OK

recognized, automounted, filemanager window opens

GADGET storage

4

OK

performance

4

NEEDS WORK

only 9MB/s from card reader which does 18MB/s with x86 laptop

SD/MMC

3

kernel

1

OK

u-boot

1

OK

performance

3

OK

17MB/s as in x86 laptop with usb card reader

serial port

2

serial kernel

1

OK

serial u-boot

1

OK

auto root login

2

OK

kernel output

2

OK

Multimedia

3

Video to Framebuffer

3

OK

Video to X11

4

OK

Video Acceleration Mem2Mem H.264

4

FAIL

Video Acceleration Mem2Mem MPEG-4

4

FAIL

Video Acceleration FullScreen

5

FAIL

Video Acceleration X11

5

FAIL

Video Acceleration X11 ZeroCopy

5

FAIL

Powermanagement

3

clock tree in debugfs

3

OK

P-states (cpufreq driver)

3

OK

C-states (cpuidle driver)

4

FAIL

test and optimise cpuidle latency using tracepoints

5

FAIL

powertop

2

OK

powerdebug

2

OK

test commands

audio

  • speaker-test to check does it generate audible sound

  • cat /proc/asound/cards to check which cards are detected

  • alsamixer to check which mixers are available and does volume change works

video

  • boot with dvi only monitor and check was it detected and does it work
  • boot with hdmi only monitor and check was it detected and does it work
  • boot with dvi and hdmi monitors and check were they detected and do they work
  • fbset to check does default monitor resolution was detected properly

  • xrandr to get list of available resolutions and list of detected displays

  • xrandr --output NAME-OF-OUTPUT --off to check does it handle turning off display

  • xrandr --output NAME-OF-OUTPUT --auto to check does it handle turning on display

OpenGLES (and EGL)

  • (you may need to apt-get install mesa-utils-extra and glmark2-es2 for these tests)
  • es2_gears to simple test of performance

  • glmark2-es2 to run a set of more coprehensive benchmarks

  • glmark2-es2 --validate to validate that the visual output is as expected

networking

ethernet

  • dhclient -v eth0 to get IPv4 address

  • iperf -c to check performance

wireless

  • iwconfig to check for presence

  • iwlist wlan0 scan to check for network detection

  • NetworkManager applet to connect to any wireless available

bluetooth

  • hciconfig to check for presence

  • hcitool scan to check for BT devices detection

USB controller

  • lsusb to check for devices connected

  • plug devices and check do they work
  • plug thumbdrive and check was it automounted
  • hdparm -t /dev/sda to check storage performance

SD

  • hdparm -t /dev/mmcblk0 to check storage performance

serial port

  • run minicom on desktop and check does messages from board appears

USB

Platform/DevPlatform/BoardSupportStatus/ComponentTestCases (last modified 2011-09-27 19:39:37)