Android Benchmark Results for the 2011-09 Engineering Cycle

About

The results in this page were conducted to compare various Linaro and Android toolchains using the Android Benchmarking Tools

logo

The toolchains used were:

The target device tested was a PandaBoard running

summary of % improvements from Andrdoid 4.4

Summary of Results

The chart to the right shows percentage improvements versus the Android 4.4 toolchain.

  • skia continues to show monthly improvments.

  • cximage all toolchains score about the same.

  • gcstone 4.5 was the only to show improvement over Android. This is slight regression from last month. However, the results from this are biased to the "Total List Alloc Time" test case.

  • gnugo Shows month-to-month improvement, but 4.5 still outperforms 4.6

  • python There was a small month-to-month regression in the 2011-09/2011-08 toolchain for 4.6.

The 4.6 graphite optimization "-floop-interchange -floop-strip-mine -floop-block" were more consistent this month than last month.

The chart on the right shows percent improvements against Android 4.4. This was done by comparing the cumulative totals for each benchmark.

Methodology

The raw data this page is based on can be found at:

The scripts used to generate the data can be found at

skia results

NOTE: smaller numbers are better

histogram

TEST

skia-android-4.4

skia-linaro-4.5

skia-linaro-4.6

skia-linaro-4.6-08

skia-linaro-4.6-O3graphite

decode_4444_(null)

0.33

0.13

0.00

0.00

0.00

decode_565_(null)

0.00

0.00

0.00

0.00

0.00

decode_8888_(null)

0.00

0.00

0.00

0.27

0.00

repeatTile_index8

185.27

184.47

186.73

188.40

184.93

repeatTile_4444

242.47

239.07

221.13

220.13

219.13

repeatTile_565

178.93

159.87

157.67

158.67

157.33

repeatTile_8888

187.20

185.07

184.73

189.13

183.20

bitmap_index8

88.13

84.93

85.20

85.80

85.60

bitmap_index8_A

179.73

177.27

176.60

178.53

178.53

bitmap_4444

65.67

74.40

66.60

66.80

64.93

bitmap_4444_A

163.07

162.93

157.47

156.33

156.87

bitmap_565

177.80

168.20

166.67

166.13

166.67

bitmap_8888

30.73

30.27

30.93

30.33

31.33

bitmap_8888_A

99.87

98.53

100.87

100.13

102.00

polygon

163.00

157.80

158.33

156.13

158.13

lines

46.87

45.07

43.80

44.67

45.47

points

10.40

10.13

10.27

10.47

7.80

rrects3

15.33

17.53

17.67

18.13

16.60

rrects1

55.27

54.80

53.73

54.60

54.73

ovals3

19.60

19.07

19.60

17.07

18.73

ovals1

64.13

63.53

63.53

63.53

63.07

rects3

1.53

1.27

1.73

1.60

1.93

rects1

10.87

9.40

11.87

11.40

11.07

Totals

1986.20

1943.74

1915.13

1918.25

1908.05

python results

NOTE: smaller numbers are better

Histogram View of Results

TEST

python-android-4.4

python-linaro-4.5

python-linaro-4.6

python-linaro-4.6-08

python-linaro-4.6-O3graphite

BuiltinFunctionCalls

676.00

697.00

693.00

651.00

661.00

BuiltinMethodLookup

501.00

462.00

499.00

496.00

496.00

CompareFloats

522.00

463.00

464.00

452.00

463.00

CompareFloatsIntegers

464.00

432.00

437.00

363.00

434.00

CompareIntegers

539.00

507.00

505.00

539.00

505.00

CompareInternedStrings

561.00

527.00

546.00

552.00

554.00

CompareLongs

502.00

448.00

489.00

517.00

481.00

CompareStrings

505.00

429.00

453.00

471.00

478.00

CompareUnicode

450.00

453.00

403.00

454.00

430.00

ComplexPythonFunctionCalls

670.00

669.00

694.00

661.00

658.00

ConcatStrings

790.00

774.00

826.00

762.00

803.00

ConcatUnicode

637.00

663.00

661.00

630.00

650.00

CreateInstances

751.00

714.00

722.00

717.00

734.00

CreateNewInstances

601.00

559.00

549.00

540.00

558.00

CreateStringsWithConcat

514.00

549.00

495.00

572.00

515.00

CreateUnicodeWithConcat

372.00

424.00

462.00

415.00

448.00

DictCreation

376.00

331.00

273.00

335.00

345.00

DictWithFloatKeys

524.00

451.00

458.00

466.00

456.00

DictWithIntegerKeys

520.00

450.00

444.00

445.00

451.00

DictWithStringKeys

486.00

424.00

434.00

431.00

439.00

ForLoops

435.00

414.00

413.00

410.00

421.00

IfThenElse

398.00

395.00

380.00

389.00

398.00

ListSlicing

459.00

569.00

568.00

566.00

566.00

NestedForLoops

542.00

483.00

510.00

489.00

532.00

NestedListComprehensions

716.00

715.00

712.00

721.00

728.00

NormalClassAttribute

546.00

504.00

532.00

496.00

549.00

NormalInstanceAttribute

468.00

435.00

500.00

433.00

531.00

PythonFunctionCalls

552.00

528.00

518.00

506.00

514.00

PythonMethodCalls

745.00

727.00

763.00

730.00

780.00

Recursion

737.00

693.00

700.00

688.00

699.00

SecondImport

771.00

757.00

839.00

829.00

840.00

SecondPackageImport

779.00

775.00

852.00

844.00

856.00

SecondSubmoduleImport

975.00

933.00

1040.00

1023.00

1062.00

SimpleComplexArithmetic

439.00

440.00

430.00

441.00

443.00

SimpleDictManipulation

544.00

490.00

485.00

480.00

496.00

SimpleFloatArithmetic

587.00

506.00

565.00

514.00

543.00

SimpleIntFloatArithmetic

409.00

375.00

396.00

392.00

394.00

SimpleIntegerArithmetic

409.00

362.00

399.00

387.00

388.00

SimpleListComprehensions

625.00

639.00

621.00

640.00

644.00

SimpleListManipulation

438.00

416.00

421.00

411.00

414.00

SimpleLongArithmetic

529.00

457.00

522.00

471.00

527.00

SmallLists

598.00

582.00

585.00

555.00

598.00

SmallTuples

579.00

550.00

594.00

580.00

589.00

SpecialClassAttribute

531.00

440.00

521.00

476.00

547.00

SpecialInstanceAttribute

605.00

563.00

600.00

590.00

650.00

StringMappings

461.00

486.00

501.00

485.00

494.00

StringPredicates

550.00

494.00

550.00

557.00

563.00

StringSlicing

543.00

515.00

537.00

337.00

509.00

TryExcept

547.00

488.00

486.00

481.00

495.00

TryFinally

640.00

572.00

588.00

582.00

575.00

TryRaiseExcept

495.00

559.00

581.00

558.00

573.00

TupleSlicing

532.00

479.00

498.00

499.00

492.00

UnicodeMappings

617.00

683.00

653.00

620.00

651.00

UnicodePredicates

640.00

561.00

595.00

601.00

605.00

UnicodeSlicing

526.00

531.00

539.00

531.00

514.00

WithFinally

712.00

737.00

775.00

763.00

760.00

WithRaiseExcept

715.00

707.00

733.00

715.00

713.00

Totals

31995.00

30961.00

31634.00

31039.00

31670.00

gnugo

NOTE: smaller numbers are better

Toolchain

Time (in seconds)

gnugo-android-4.4

134.20

gnugo-linaro-4.5

123.38

gnugo-linaro-4.6

123.98

gnugo-linaro-4.6-08

123.95

gnugo-linaro-4.6-O3graphite

123.80

gcstone results

histogram NOTE: smaller numbers are better

TEST

gcstone-android-4.4

gcstone-linaro-4.5

gcstone-linaro-4.6

gcstone-linaro-4.6-08

gcstone-linaro-4.6-O3graphite

Total Array Alloc Time

147.00

133.00

159.00

162.00

165.00

Total Array GC Time

27.00

27.00

24.00

27.00

24.00

Total List Alloc Time

4019.00

3979.00

4111.00

4120.00

4033.00

Total List GC Time

703.00

705.00

706.00

703.00

709.00

Totals

4896.00

4844.00

5000.00

5012.00

4931.00

cximage

NOTE: smaller numbers are better

Toolchain

Time (in seconds)

cximage-android-4.4

40.00

cximage-linaro-4.5

40.57

cximage-linaro-4.6

40.24

cximage-linaro-4.6-08

40.00

cximage-linaro-4.6-O3graphite

40.31

Platform/Android/AndroidToolchainBenchmarking/2011-09 (last modified 2011-10-21 19:02:42)