@misprintedtype COOL, WHAT NOW? IM OFFLINE - we <3 GOTOBerlin - - - PowerPoint PPT Presentation
@misprintedtype COOL, WHAT NOW? IM OFFLINE - we <3 GOTOBerlin - Hello there! How are you? Are you tired or still good ? Did you enjoy your first day so far? Cool! Okay So, lets do this. Welcome to my talk about best practice in offline
@misprintedtype
COOL, WHAT NOW?
Hello there! How are you? Are you tired or still good? Did you enjoy your first day so far? Cool! Okay… So, lets do this. Welcome to my talk about best practice in offline first and the learnings we gathered in the past few years.
OLA GASIDLO
Lead Developer Co-Organizer CSSConfEU
My name is Ola Gasidlo. I am a lead developer at Styla, Co-Organizer of CSSConfEU and organizer of various conferences and meetups and even more… Today I’d love to tell you a bit about the actual state of offline first, how our point of view changed in the past few years and how its adapting to the real world.
@misprintedtype
Trigger Warnings
death / pandemic animated gifs
Before I start, I’d like to mention in this talk I’ll speak a pandemic and death, but there won’t be any kind of graphical display of those topics. Plus there will be animated gifs.
@misprintedtype
Trigger Warnings
death / pandemic animated gifs
So, if you’d rather leave, because you won’t feel comfortable with this content… I fully understand. I am happy to chat with you after the talk and give you a summary of the content. You still have time to leave, if you’d like to until the first gifs. Let’s start then.
@misprintedtype
The internet, it turned 25 this year. Amazing times we all live in…
@misprintedtype
In this time, our relationship with the internet changed. A lot. We do different things, with different devices than 25, 10, 5 years ago... and it is a part of our everyday life! We rely on our devices... the internet, but our needs changed.
@misprintedtype
Three years ago, when the Offline First movement got broader attention, I gave my first talks about the concept. That was in twenty-thirteen. What was our idea back then…? Why did we need the offline first concept so badly?…
@misprintedtype
First thing was the rise of the mobile devices and their usage. There are countries, where people do not even use desktop devices at all anymore. When you think about how cheap a phone is in comparison to e.g. a laptop, it totally makes sense, right? A mobile device is mostly enough, if you consider what the user needs it for.
@misprintedtype
There was also the issue of no constant connectivity or poor network coverage. Here offline first felt like a huge enhancement to your application and adaption to our new behavior.
@misprintedtype
Three years later, in twenty-sixteen, what do you think..? Do we still need the offline first concept?
@misprintedtype
Tell me… (by raising your hand)
@misprintedtype
After seeing offline first in the wild, we’ve realized… it’s not just about a special use case or rise of the mobile devices. It’s about the whole approach… It’s so much more.
Jake Archibald @twitter
As Jake Archibald once said… „Offline first“ means getting on screen without a network request…
Jake Archibald @twitter
…The more that’s there before the network, the more offline first it is. This means, every web app should work without a connection after the first visit and just use the network for getting / setting new data. Everything else is online-first.
@misprintedtype
Remember me asking you if offline first is a progressive enhancement to your application? Like 3 slides ago? What if I tell you, you are actually…
@misprintedtype
…wrong! -> 1, 2, 3 This might sound confusing… Why is the offline first concept not progressive enhancement? Actually it is the other way around… Let’s take a step back and see, why do we need applications?
@misprintedtype
Normally we use applications for our everyday life, to make our everyday life easier, right? Like…
@misprintedtype
Checking in with family members and other loved ones abroad or maybe just text your partner during the day.
@misprintedtype
Using Slack, IRC etc. at work, especially in remote teams.
@misprintedtype
@misprintedtype
We also need applications for tracking data. Did I drink enough during today? How many steps did I make? How much money did I spend this week? But also…
@misprintedtype
How can I fix my bike or my washing machine? How do I translate this or even… „Did I spell this right?“
@misprintedtype
@misprintedtype
Posting selfies from your last vacation, tweeting your thoughts etc. Checking in symptoms when you are sick…
@misprintedtype
Or even check the internet why your kids act out.
@misprintedtype
AND… most important thing…
@misprintedtype
cat.gifs… (1… 2… 3…)
@misprintedtype
—> love. So, when we look at those points… Communication, information, empowerment and love, this all sounds like something we all have in common right?
@misprintedtype
When we break it down, it all comes down to help satisfy our very basic, humane needs for safety, belonging, esteem and self-actualization. Maslow defined all those points in his „Hierarchy of needs“ already in nineteen-fourtythree.
@misprintedtype
So since when is helping people to fulfill basic needs, progressive enhancement?
@misprintedtype
// drauf zeigen… sounds familiar right? But we love to sell it that way, so developers jump on the progressive enhancement train and just redo their apps every 6 weeks in the new, cool framework or
@misprintedtype
// drauf zeigen… nech? To be honest… I am so tired of half done applications, which are released out in the wild without any kind of documentation, because someone wanted to refactor their app in the „new cool stuff"…
@misprintedtype
// drauf zeigen… -> show quotation marks… Or super slow apps, b/c of bad planing and messy architecture.
@misprintedtype
RETHINK! We do not need a new UBER or Twitter… REALLY. Trust me! We tend to forget that THIS, our JOB is all about humans, people… not frameworks, tech or a design pattern, or the discussion about semicolons.
@misprintedtype
Don’t build a town, when you need a house. Just because you can! Especially because usually you end up with the „Gorilla Banana Problem“.
Joe Armstrong
The Gorilla / Banana problem: Every time you use a whole infrastructure like a framework to solve one problem, you get all this implicit environment that they carry around with them. It’s about using the right tool for a job. Without building a town, when you just need a house. Just because you can.
@misprintedtype
It’s all about keeping things simple, but solid. Basic, but optimized. This is why I think, putting offline first in the progressive enhancement slot is pure BS.
@misprintedtype
Do you know, what the progressive enhancement part of your application is? The network. You might ask yourself… Why should I care? My apps are working…
@misprintedtype
Are they? (1… 2… 3…) Ever checked your application without or with a bad connection?
@misprintedtype
How productive are you when e.g. just one service like GitHub is down? Not to mention the office WiFi? How easy is it to communicate with your friends at a concert, when everyone is sending a live stream of the concert via their phone?
Bruce Lawson
And besides us - with good connectivity situation blessed people, there are others…
@misprintedtype https://speakerdeck.com/benschwarz/vanity-performance-metrics-looking-beyond-the-numbers
Just 43% of the population has access to an internet connection. And just a small amount of them to a fast and steady one.
@misprintedtype https://speakerdeck.com/benschwarz/vanity-performance-metrics-looking-beyond-the-numbers
And data is expensive… As you can see here on this list from Ben Schwarz’s talk about performance metrics. Germans would have to work less than one hour for 500 MB, but people in India about 17 hours…
Bastian Albers
Let’s assume you have a 500MB plan…
Bastian Albers
So you could visit almost 7 sites per day during that month.
Bastian Albers
Or watch 22 minutes of talks… So basically ONE talk per month and your data would be gone.
Bastian Albers
https://www.whistleout.com.au/MobilePhones/Guides/Mobile-broadband-usage-guide
Or you could make your others really happy and finally write ALL the mails that you are procrastinating… Without attachments of course.
@misprintedtype
Our applications should provide the base and work offline per default. The network is the one to enhance our application with new, outside data.
@misprintedtype
No matter which technology stack we use… Peer-to-Peer network, direct connection to the server or a USB-Stick, whatever works to transfer the data. It’s not important HOW you transfer the data or application on a device. What is important is, we have saved it all locally and enhance the sync through the network.
@misprintedtype
Not sure, this is true? Let’s check out what is happening in the real world and how others solve their offline issues.
@misprintedtype
Let’s start with… the Router…
@misprintedtype
https://ingevald.files.wordpress.com/2009/02/sweden-river-fog-morning-looking-at-sun.jpg
Imagine… A beautiful morning. You’ve decided to work remotely and are sitting in an „internet cafe“. You’d love to check your mail? Cool! Connect your laptop to the router. Open your email app and send the request to get your mails. Close the laptop and come back in the evening… wait, WHAT!? Why you might ask?
@misprintedtype
https://ingevald.files.wordpress.com/2009/02/sweden-river-fog-morning-looking-at-sun.jpg
Well… There is no „physical“ connection to the internet like a cable that the internet cafe is using. They just have a router that saves your request and connects via bluetooth to a…
@misprintedtype
bus! It syncs the requests and off it goes. Literally. The bus arrives at a stop after half of the route to the hot spot with physical connection and syncs with another bus. Also exchanges not just the data, but also the passengers and luggage.
@misprintedtype
The buses sync via bluetooth previous requests and results. Both buses return. Literally and yeah… you get your emails.
@misprintedtype
(SNC - Service) Using NAT zones, DTN routing && DakNet network. Saami Network Connectivity
The tech behind the SNC Service is a combination of NAT zones, DTN routing and the DakNet network. (Not to be confused with the DarkNet.)
@misprintedtype
Learnings (requests & assets)
So, what are our learnings here?
@misprintedtype
Check your requests!
Actually check your requests, because beside you should be aware of what your are actually doing… Do you need 20 requests or would actually just 2 be enough? Please serve uglify and minified files. Do you really need that 5MB background image on mobile?
@misprintedtype
beware of Lie-Fi
You should be aware of Lie-Fi, which is the state, where your device checks your connectivity situation, assures that you are online (pinky promise), but there is actually no data passed through.
@misprintedtype
data queries with bandwidth?
We could go with checking the bandwidth after the very fist ping right? Yes, that could be a thing! And we could develop something similar to media queries and device sizes but just with bandwidth. So we can load assets per detected TRUE bandwidth.
@misprintedtype
Great idea, right?! But the bandwidth can differ per request, even per package, especially when you are moving, so e.g. are on a train.
@misprintedtype
data queries with bandwidth
We would need to ping not just per every request, but constantly. You can’t rely on requests to come back. Please add a timeout to kill your request or make the app work fully independent of requesting data.
@misprintedtype
enhance assets with network
This is why we need to separate updating cycles of the assets and the data. The update of assets is less important than actually syncing data.
@misprintedtype
save basic set of UI locally
Save a basic set of your UI assets locally (to go easy on your storage). Enhance it with e.g. web fonts, images etc. Whatever you need… Offline should be the default state and we need to make the bandwidth assumption by the device we detect.
@misprintedtype
keep the caches clean
Take responsibility for the caches and keep them clean. What could be better to use to manage your caches, than Service Worker. No intro in Service Worker today. Sorry. Just one thing…
@misprintedtype
If you want to use the AppCache for quite a bit, because browsers are dropping the support hard, but also for migration from AppCache to ServiceWorker, please check out the JakeCache.
@misprintedtype
SVG over IMG
Images… Hard to deal with within the offline context… Saving images in the in-browser storage? Good way to go, but it is still very painful because of storage limitations and very different storage limit implementations. Porting decorative images, like icons and logos to SVG and CSS is a great way to go to cut off a bit of the overhead.
@misprintedtype
base64 over IMG
Encoding assets in base64 and storing the String in the in-browser storage of your choice could be a way to go. For example, saving fonts like that and loading them straight from the storage, improves the performance a lot. This is even possible for images, like a logo. But please, just do not do this for all of your images. Don’t build the town, build the house. Remember? The strings are huge as we all already know. And in-browser storages are is limited.
@misprintedtype
base64 is expensive && heavy
Also decoding base64 on the fly is a heavy task. So it slows down your site. Especially in combination with DOM-manipulating frameworks like React. You’ll also find yourself running into race conditions here.
Let’s check out the next use case… It is a bit heavier, as already mentioned. Just to let you know in advance.
@misprintedtype
@misprintedtype
Twenty-fourteen and twenty-fifteen, the ebola pandemic in parts of West Africa was a really scary issue.
@misprintedtype
I was involved in a project by the NGO eHealth Africa that build two applications to help stop the pandemic. The first application to solve one of the issues was… providing information for the population, because information is the key to prevent a pandemic to spread or kill.
@misprintedtype
The process was… The population was informed about the number they could call to ask about ebola symptoms, to report someone sick or even dead. The call center tracked the calls with the application we’ve build, collected all the provided information from the caller and sent out a team, if needed.
@misprintedtype
They added information through the application like the name, location and additional information. Also added the phone number of the caller, so they could call back for any further questions. Data analysis and filtering was really easy thanks to the application.
@misprintedtype
They fought with issues like unexpected and constant power loss. The generators jumped in after some time, but we could not afford to lose data. Just one case could have caused another outbreak and many deaths.
@misprintedtype
first learning (data)
Our first learning here was…
@misprintedtype
Decouple server && client
Decouple server and client, to serve the needs of a eventual contingency.
@misprintedtype
build a task / message system
Build in a task and message system, that can act independently from the connectivity situation. For example… for data CouchDB, especially in combination with PouchDB proved their usability in the real world really, really well.
@misprintedtype
CouchDB a NoSQL database that replicates, which is exactly what we want for sync. CouchDB comes with a suite of features, such as on-the-fly document transformation and real-time change notifications, we need for our message and task system, but also for a decoupled architecture.
@misprintedtype
PouchDB is an open-source JavaScript database inspired by CouchDB that is designed to run well within the browser and store all your data locally while offline.
@misprintedtype
It sync’s with CouchDB thanks to the CouchDB replication protocol, when the application is back online.
@misprintedtype
save data documents locally
With PouchDB we actually have documents which are stored locally. So we won’t not flush the caches during by accident.
@misprintedtype
prevent merge conflicts
One of the biggest issues with syncing data are merge conflicts. Because PouchDB and CouchDB speak the same protocol and sync automatically when we are back online and we do not need to take care of it.
@misprintedtype
The second application we’ve build, was to actually track patient data in the field. After examining the infected patients, who had to stay in quarantine for 21 days. The team added the data to a general database through the application plus got lab results from several different entry points and very different file types.
@misprintedtype
The data was imported, saved in a global database and synced all across all kind of devices. This all sounds super messy, right? What do you do with a mess? Well, I am a parent. So I am used to cleaning up a mess…
@misprintedtype
second learning (schema)
Our second learning here was…
@misprintedtype
define flexible data schema
We added an organization system. An interface to map the data in one defined schema. The schema was flexible enough to be extended, if needed… Thanks to the NoSQL database behind it.
@misprintedtype
test data schema constantly
Data was imported automatically when it arrived (mostly per mail) or we added data on the fly. Constant automated testing and a high test coverage for the importer and the interface was a must.
@misprintedtype
prevent merge conflicts
The schema and the testing helped us a lot here to prevent merge conflicts before the get go. CouchDB and PouchDB helped us to take care of the sync, merge and flexibility.
@misprintedtype
prevent merge conflicts
Big bonus also? The schema also helped us with making filter and analyze the data, so visualizing it was really easy.
@misprintedtype
10/2014 12/2015
We helped to curtail the pandemic, because of clean data structure and constant testing of the applications. If anyone tells you that those things don’t matter or you don’t have any time for those, you now know what to reply. Almost lunch… Guess, well picked theme for the next use case. Sorry in advance…
@misprintedtype
@misprintedtype
We all love to order food, right? Did you ever check the localStorage when ordering? Just to see, if they save the cart items in it? A friend of mine did this once and went bit further out of curiosity. He was wondering if they are actually validating their checkout on the server.
@misprintedtype
So he manipulated the checkout and subtracted one cent, send it out and waited. The pizza came. And no, there was no server side validation.
@misprintedtype
Learnings (security)
What do we learn here?
@misprintedtype
validate on server
In contrast to the limited possibilities of our use case before, this company could have validated the data on the server.
@misprintedtype
validate on server
But this is sometimes not possible, especially when everything needs to work without any requests. Security is still a very new topic to offline first and not many tools are provided yet. And I’d be happy to chat especially about that topic after my talk.
@misprintedtype
CORS <3
Although many of us seem to hate CORS sometimes, it provides already a great set of rules to make web applications more safe. But it does not solve the issues with the in-browser storage manipulation.
@misprintedtype
encrypt local data
One way to go would be to encrypt your data. When you use PouchDB, you can add CryptoPouch, which encrypts your Pouch documents. Big downside here is, the document get’s encrypted, but not the ID. Which can lead back to security issues. But at least you can’t change the data anymore in the console.
@misprintedtype
When you want to encrypt the data, you might want to log in. Which does not work without a request to the server, right? There is a way to use tokens in the client.
@misprintedtype
JSON web tokens are an open industry standard method for representing claims securely between two parties, which is exactly what we want! It makes decoupled and secure sync and communication possible.
@misprintedtype
BONUS LEVEL!!! Because this is pretty much the most creative use case I’ve ever heard to solve the offline issue.
@misprintedtype
The Rally.
@misprintedtype
https://en.wikipedia.org/wiki/Mongol_Rally#After_the_rally
Anyone of you heard about the Mongol Rally before? It’s an unconventional rally where people start from London or the Czech Republic and drive down to Mongolia. The main intention of this rally was a „donate your car“ program where every car that makes it to Mongolia, can be donated without import duties.
@misprintedtype
http://www.minicong.com/lo-tec/
2009 a team from Spain took part in the rally. Nothing special at first glance… Well. The issue at the rally was, they had to cross a LOT of countries. With different receptions, different payment models for data volumes etc. So they needed a way to communicate with their loved ones, so they could say… „We are alive and well.“
@misprintedtype
They built a system, that could encode an ASCII message in 8 bit binary code, ran the bits through 8 phones, that actually called just one number.
@misprintedtype 1 2 3 4 5 6 7 8 1 1 1 1 1 1 1 1 1 1
To save money, they didn’t establish a full connection. It was mainly about letting it ring. The short ringing was a zero, the long ringing was a one. The other end, checked the numbers, which were set for order of bits and decoded the binary back in ASCII. Then it sends out message to loved ones. <3 Mind blowing, right? Yeah, people… :D
@misprintedtype
Learnings
What do we learn here?
@misprintedtype
Rethink how to use technology, by thinking about the issue. Not about the tools.
Read quote… Unidirectional communication. Very basic. They fixed it all already in the beginning by thinking about the real issue and not the solution first.
@misprintedtype
You rock!
Okay, folks… Looks like our time is up. I hope, you had a great time and thank you for listening. If you have any questions, let’s talk. Otherwise please reach out via twitter or find me here somewhere. And remember…
@misprintedtype
It’s not about frameworks… or design patterns or tools…
@misprintedtype
… it’s about humans!
@misprintedtype
we <3 you!
Thank you.
@misprintedtype
And please remember to rate this session OR give feedback in person. Thank you!