New Features in BulkTracker pkgsrcCon 2015 Benny Siegert - PowerPoint PPT Presentation

Feb 09, 2023 •1.24k likes •1.46k views

New Features in BulkTracker pkgsrcCon 2015 Benny Siegert bsiegert@NetBSD.org Outline 1. Introduction 2. Statistics 3. New features 4. Embracing JavaScript and JSON 5. Future work 6. What is BulkTracker? Web app to track pkgsrc bulk

New Features in BulkTracker pkgsrcCon 2015 Benny Siegert ‹bsiegert@NetBSD.org›
Outline 1. Introduction 2. Statistics 3. New features 4. Embracing JavaScript and JSON 5. Future work 6.
What is BulkTracker? Web app to track pkgsrc bulk build status. “Which package fails to build on which platform?” Built in Go, running on Google App Engine. http://bulktracker.appspot.com/ github.com/bsiegert/BulkTracker
How does it work? Subscribed to mail to pkgsrc-bulk@. - Parses incoming reports, downloads “machine- readable” version. - Generates 1 record per build + 1 per package result.
Some statistics 768 builds, ~4M individual package results 800M of data (plus 12G of indexes …)
New feature: latest build per platform Collapse repeated builds into one: - Default “builds” view on home page only shows one build for each {branch, platform, user} Also the default set for package results.
New feature: package results Enter category/pkgname, get build results Currently: for latest build for each platform Need UI for selecting builds / versions to show Originally planned: grid view ‒ useful?
Embracing JavaScript Most pages don’t use JavaScript (just DataTables), all logic is in Go code + templates. Package results is static HTML + JSON. It turns out that some things are easier in JavaScript. (Using the DataTables API.)
JSON API = clean backend Clean separation of logic and presentation. Can store the entire result in memcache. Shared memcache on App Engine is free! Go can directly marshal structs ↔ JSON. If people are interested, could become an API.
Thinking about the data model (too) many writes to the datastore. Use Cloud SQL? (probably more expensive) Use blobstore and store all build details as a single blob? - will make cross-correlations slow.
Remaining pain points What if fetching machine-readable report fails? UI needs improvement. - surface builds with no detail - make “retry index” feature more interactive
Other requirements PRD in the wiki for such a tracking tool, not all of the requirements have been addressed. - keep track of updates to pkgsrc [how?], surface updated packages that have not been built - annotate build failures with explanation - “results from packages you maintain”
Conclusion http://bulktracker.appspot.com/ Looking for people interested in joining or sending pull requests!

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Supervised Learning Given: a set of inputs features X 1 , . . . , X n a set of target features Y 1

Supervised Learning Given: a set of inputs features X 1 , . . . , X n a set of target features Y 1 , . . . , Y k a set of training examples where the values for the input features and the target features are given for each example a new

323 views • 18 slides

Efficient visual search of local features Cordelia Schmid Bag-of-features

Efficient visual search of local features Cordelia Schmid Bag-of-features [Sivic&Zisserman03] Query Set of SIFT centroids image descriptors (visual words) sparse frequency vector Bag-of-features Harris-Hessian-Laplace processing

574 views • 33 slides

New Features of Net-FM v. 3.3 & 3.3.1 ICC 2016 ICC 2016 New Features of Net-FM v. 3.3 &

New Features of Net-FM v. 3.3 & 3.3.1 ICC 2016 ICC 2016 New Features of Net-FM v. 3.3 & 3.3.1 presented by Anthony J. Bunker Software QA & Documentation Manager October 20, 2016 Slide 1 Introduction ICC 2016 ICC 2016 At

374 views • 15 slides

Image Features: Detection, Description, and Matching and their Applications Image

Image Features: Detection, Description, and Matching and their Applications Image Representation: Global Versus Local Features Features/ keypoints/ interset points are interesting locations in the image. Features serve to give a new

235 views • 22 slides

reliable innovation Main features Main features Lower part detachment Concept advantages

SIRE-NN Chocolate enrobing reliable innovation Main features Main features Lower part detachment Concept advantages Cantilever design for easy access (cleaning and maintenance) Detail features Detail features Pilot plant Compact

600 views • 13 slides

SI485i : NLP Set 12 Features and Prediction What is NLP, really? Many of our tasks boil down

SI485i : NLP Set 12 Features and Prediction What is NLP, really? Many of our tasks boil down to finding intelligent features of language. We do lots of machine learning over features NLP researchers also use linguistic insights, deep

326 views • 18 slides

Image Features Sanja Fidler CSC420: Intro to Image Understanding 1 / 64 Image Features Image

Image Features Sanja Fidler CSC420: Intro to Image Understanding 1 / 64 Image Features Image features are useful descriptions of local or global image properties designed to accomplish a certain task You may want to choose different features

1.09k views • 71 slides

Mark Pagel pags@cray.com New features in XT MPT 3.1 and MPT 3.2 Features as a result of

Mark Pagel pags@cray.com New features in XT MPT 3.1 and MPT 3.2 Features as a result of scaling to 150K MPI ranks MPI-IO Improvements(MPT 3.1 and MPT 3.2) SMP aware collective improvements(MPT 3.2) Misc Features (MPT 3.1)

694 views • 33 slides

Considered ELA Features Sunday, July 15, 2018 9 / 20 Considered ELA Features Meta-Model

Considered ELA Features Sunday, July 15, 2018 9 / 20 Considered ELA Features Meta-Model Features: fits linear and quadratic models (with and without pairwise interaction e ff ects) to the data extracts information from these models, such as

436 views • 7 slides

The e-Application Form Main features Main features ENI CBC Med Programme - Managing Authority

The e-Application Form Main features Main features ENI CBC Med Programme - Managing Authority Regione Autonoma della Sardegna 1 Intro Focus on the online Application Form (eAF) main features namely: Result based management applied to the ENI

562 views • 23 slides

http://cs246.stanford.edu Input features: N features: X 1 , X 2 , X N A Each X j

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Input features: N features: X 1 , X 2 , X N A Each X j has domain D j X 1 <v 1 Categorical: C Y= 0.42 D j = {red, blue} X 2

508 views • 40 slides

Part I Georgia Milestones: Unique Features Some disabilities All schools MUST Features

Part I Georgia Milestones: Unique Features ***Some disabilities ***All schools MUST Features include: test some can be tested pencil- paper. students online. inclusion of constructed-response items in ELA and mathematics, in

784 views • 24 slides

Image Features Sanja Fidler CSC420: Intro to Image Understanding 1 / 1 Image Features Image

Image Features Sanja Fidler CSC420: Intro to Image Understanding 1 / 1 Image Features Image features are useful descriptions of local or global image properties designed (or learned!) to accomplish a certain task You may want to choose di ff

941 views • 70 slides

FACULTY MODULE This power point will guide you through the features and their usage Features:

Student Accessibility and Accommodation FACULTY MODULE This power point will guide you through the features and their usage Features: 1. Electronically sign decision letters 2. You will be able to see students who have approved

744 views • 17 slides

New features of 660 MW Units Turbine Maintenance Sipat Super thermal power project New features

New features of 660 MW Units Turbine Maintenance Sipat Super thermal power project New features of 660 MW units Constructional features Operational features Lube oil system Governing system Feed water system Turbine

1.26k views • 98 slides

features and potentialities of the JOREP database features and potential uses in the studies on

Joint and Open REsearch Programmes: Exploring JoREP 2.0: features and potentialities of the JOREP database features and potential uses in the studies on Europeanisation of research activities CERIS-CNR, Rome November 27-28, 2014 IRCrES-CNR,

303 views • 7 slides

Mass Digitization on Demand Automation and Terrible Metadata We Digitize for Remote Requests

Mass Digitization on Demand Automation and Terrible Metadata We Digitize for Remote Requests 1-3 requests for scanning per week Performed by student assistants Simple Fact: The most costly part of any traditional digitization project is

355 views • 12 slides

Overview CS 446 What is machine learning? Machine learning : study of computational

Overview CS 446 What is machine learning? Machine learning : study of computational mechanisms that learn from data to make predictions and decisions. 1 / 21 Application 1: image classification Birdwatcher takes photos of birds,

1.08k views • 83 slides

How & Why is the Arctic Changing? -> Integrate IASOA datasets and experts in sustained

Inter ternational A Arctic S System ems f for O Obser erving t g the A e Atmospher ere e (IASOA) ion A A Por ortal f l for D Dis iscovery, a a Pla latform f for Pa Pan-Arctic ic C Collab labor oration

493 views • 18 slides

Kotlin Serverless Framework Vladislav Tankov What is serverless? cloud-computing execution model

Kotlin Serverless Framework Vladislav Tankov What is serverless? cloud-computing execution model , in which the cloud provider runs the server and dynamically manages the allocation of machine resources Wikipedia How are serverless

995 views • 46 slides

CS 6453: Parameter Server Soumya Basu March 7, 2017 What is a Parameter Server? Server for

CS 6453: Parameter Server Soumya Basu March 7, 2017 What is a Parameter Server? Server for large scale machine learning problems Machine learning tasks in a nutshell: Feature (1, 1, 1) Training Extraction (2, -1, 3) (5, 6, 7)

520 views • 21 slides

INFO-4604, Applied Machine Learning University of Colorado Boulder August 28, 2018 Prof. Michael

INFO-4604, Applied Machine Learning University of Colorado Boulder August 28, 2018 Prof. Michael Paul People Professor Michael Paul 4th year at CU 2nd year teaching this course mpaul@colorado.edu Teaching Assistant: Arcadia

473 views • 15 slides

Large-Scale Data Engineering Introduction to cloud computing + Hadoop, HDFS & MapReduce

Large-Scale Data Engineering Introduction to cloud computing + Hadoop, HDFS & MapReduce event.cwi.nl/lsde2015 COMPUTING AS A SERVICE event.cwi.nl/lsde2015 Utility computing What? Computing resources as a metered service (pay

1.15k views • 77 slides

The Poisson Arrival Process CS 70, Summer 2019 Bonus Lecture, 8/14/19 1 / 22 Poisson

The Poisson Arrival Process CS 70, Summer 2019 Bonus Lecture, 8/14/19 1 / 22 Poisson Distribution: Review - neg integers non Values: X " Parameter(s) : rate " , a P [ X = i ] = e- . it . x E [ X ] = Var[ X ] = x 2 / 22

444 views • 22 slides