The Future of the Open Programming Symposiums

On February 17, 2024 By scottcollisIn UncategorizedLeave a comment

Started by Johnny Lin, the Python Symposium (Symposium on Advances in Modeling and Analysis Using Python now the Symposium on Advances in Modeling and Analysis Using the Programming Languages of Open Science) has been a home for showing the community of the American Meteorological Society how we do our science. A place to show code and nurture the next generation of contributors in the weather and climate enterprize. I (Scott) took over leadership after the 2017 meeting in Seattle chairing the 2018 meeting in Austin (with an amazing keynote from Matt Rocklin!) . Since then my career has, well, taken off and it is time for new leadership and a reimagining of our symposium. At the 104th annual meeting in Baltimore I announce my plan for the 105th meeting in New Orleans to be the last symposium I lead. I hope to have a cadre of new leaders help me with the 105th meeting. We will be holding a virtual meeting in May to recruit and discuss the future of the symposium. Please sign up and express your interest HERE. This will help us decide the future of leadership for the 106th meeting in Houston and beyond.

Leading the symposium has been a pleasure! Each year after leading it I feel rejuvenated and keen to do it again. It is important to pass on the torch to new generation who can innovate and take our symposium in a new direction.

Fallen For Fall

On October 16, 2022 By scottcollisIn Cycling, General Weather Geekery, Hobbies, Science Communications, UncategorizedLeave a comment

Louise and I were having a discussion about seasons, economics and.. well.. humanity. I am so lucky to be with someone who is so deeply cerebral and who’s proclivities (going deep and philosophical on a subject) match my own.

I posit Australians who have never experienced boreal fall don’t understand just how seasonal and how much Americans revel in seasonality. We were discussing how the fleeting nature and rarity of spring and fall have spawned many celebrations of its passing. While the depths of winter can seem interminable (but not without its special beauty) and the dog days of summer are long, fall and spring seem fleeting and folks seem keen to soak it up.

Spring and fall are also very colorful. The monochrome of winter giving way to the blues, pinks and purples of spring and the green of summer giving way to the flames of fall (Autumn for those of the Commonwealth). The district change in the weather is also a cause for celebration. The first sweet warmth in March and a crisp morning in October.

Strangely due to some work and mental health issues, fall to spring are my peak training times. Cycling makes me happy and being happy makes me cycle. I get busy in summer and forget the love of the bike. Come fall I start thinking to spring races and get my behind into gear. I then curse myself for wasting away the warm 5am starts as I shiver in sub-freezing conditions before work (or wimp out and head to the pain cave to Zwift). Ironically fall is my rebirth, perhaps the reason why it holds a special place in my heart.

This year I am targeting an earlier ride than Barry so I am going to ramp up on the Forge Fat Bike race. So the training has started! Well and truly fallen for fall!

Community Research on Climate and Urban Science

On September 7, 2022September 8, 2022 By scottcollisIn Projects, Uncategorized, urban science1 Comment

The day has finally come! As is usual with projects funded by the government we find out many weeks ahead we have been funded and are under embargo as the details are sorted out. Let me start by saying to my fellow scientists who were not awarded: I feel your pain. It is unpleasant, to say the least, to work so hard on a vision and be told you can not carry it out (yet).

Charlie Catlett showing a Sage node to Dr Berhe, Director of the Office of Science with Paul Kearns, Director of Argonne National Laboratory.

Community Research on Climate and Urban Science or CROCUS is an Argonne led response to a call for proposals by DoE’s Office of Science. In a nutshell; through modeling and measurements, we will shine a light on climate relevant atmospheric science at the street level IN CHICAGO!

I will be leading the Measurement Strategy Team. We will be doing two very exciting things: Building a network of AI enabled sensors across Chicagoland. This is the Chicago Micronet. And we will be running a series (three) of field campaigns aimed at understanding the urban science behind the three climate elements that impact the people of Chicago: Heat, Water and Air Quality.

Bad air is a result of industry transport and energy, water moves and often where we tell it to through urban hydrological systems and heat KILLS. Heat kills more than tornadoes and is the nation’s most deadly weather phenomena.

Waggle: The cyberinfrastructure that will enable CROCUS!

For the field campaigns (and I almost giggle with excitement) we have partnered with the University of Washington at St Louis, a leader in understanding the science of aerosols (tiny particles, one millionth of a meter across). The University of Wisconsin–Madison Space Science and Engineering Center and their Portable Atmospheric Research Center (SPARC). And The University of Illinois at Urbana-Champaign Flexible Array of Radars and Mesonets (FARM) which includes the famous Doppler on Wheels (DOW)!

This will culminate in the largest, most inclusive, most open, and, most comprehensive study of an urban environment ever on the planet!

Stay tuned for more news including how we will work with partners like Blacks In Green. To our friend in the community who did not fare as well: I feel for you. But, I am personally dedicated to make CROCUS open and welcoming. Come to Chicago, collaborate with us and we will have so much fun equipping the communities in Chicago with the knowledge they need to fight and prepare for climate change.

Curating Weather Simulation Data. Earthcube Workshop in North Dakota.

On July 27, 2022 By scottcollisIn conferences, Open Science, Projects, Travel, UncategorizedLeave a comment

“Simulation outputs are important but that does not mean we save them forever” – Gretchen Mullendore

This week I have been attending a workshop on data curation (a key part of open science) specifically on developing guidelines on the data produced by weather and climate simulations. Open science is better science! But a blanket “you must save and provide all data” is not only onerous (especially for underserved institutions) but not what is needed for reproducibility and reusability.

So many great minds focused on open science.

First, this post are my thoughts and do not, necessarily, reflect the views of attendees and organizers. There will be a report. There has been a lot written about measurements and measurements can no be recreated. Model data, to a degree, can be regenerated. By sharing workflows those with the appropriate resources can run the models on provided initialization and configuration data. Furthermore the sharing of workflows allows the exploring of the robustness of conclusions to assumptions (sensitivity) and the reuse of the workflow to address new science questions.

I really enjoyed the discussions and applaud the team’s focus on designing rubrics as it brings the conversation up a level and enables the clear measurement of the efficacy of solutions. It was also great seeing a huge diversity in the career stage and “flavor” of participants. We had data creators, curators, representatives from three publishers (AGU, AMS and PLOS), data scientists and more!

Susan from the University of Michigan on data curation.

Also, fittingly, lots of discussions around equity. Open science is better science. Journals are increasingly requiring data to be made available (even FAIR) which can create a burden to institutions without the physical and/or workforce to meet these requirements. There have been discussions of carving out exceptions for underserved communities. My perception is that the community here at the workshop pushed back hard against that idea as, as aforementioned, open science is better science. Rather we need to equip those institutions to meet the open science requirements.

Lots of discussions on just how much data should be required to be made available to be open and how long it should be curated for. Again a focus on designing rubrics to guide the process. The focus should be on the goal and be flexible to aid the scientist in achieving open science and reproducibility and also allow the society driven journals in meeting the aspiration of is members.

A nice atmosphere and a nice atmosphere!

It was great to be back in Grand Forks. The University of North Dakota is a great institution that, in the atmospheric science, punches way about its weight. Two of our recent three hires had a background at UND and I very much enjoy my collaborations with the team there. It was also very nice to be there during a dry cool air outbreak in summer rather than a frigid cold air outbreak in october!

Planning around the mountains

On July 31, 2021 By scottcollisIn Destinations, Travel, UncategorizedLeave a comment

It’s monsoon season in the mountains. Warm August sun heats the high peaks where the atmosphere is cold. This leads to unstable air and clouds and, if the upper atmosphere is not too dry, storms and rain. And the atmosphere is not to dry at the moment. This is good on so many levels, primarily it allows the high country to flourish and blossom. And it puts a damper on the fire danger. For visitors and locals alike, it means one needs to plan around the weather. I am no expert in Leadville local scale meteorology but through a 5-day in-situ study it seems that clouds build shortly after 8am, get taller and spread on ranges west and east of the Arkansas headwaters valley at around 10am, precipitation can start at around 11am with storms moving in after noon. The NWS had upped its wording for today (Saturday the 31^st, last day of July) with stronger terms. Good forecast as it validated nicely, albeit later that forecast which seems to be a recurring theme, or, more on point, accurate with the probabilities assigned (a topic for a later post perhaps). Well, enough weather geekery! Seeing the forecast and discussing priorities with Louise and River we decided today was Kayak Day and we needed to go as early as possible. A call to Mt Elbert Kayak (can’t recommend highly enough) and we were booked in at open at 8. We made it to Twin Lakes at 7:30am, enough time for a Coffee and breakfast out of a WV camper. Delicious and a great view. Paul at Mt Elbert Kayaks was amazing. He made safety fun! He showed us the ropes (almost literally) and got our rental watercraft strapped to the Subaru. We were doing the full-on mountain life! The benefit of being up and out so early is the water was glass. The plan was to launch at the “Red Rooster” (no idea) boat ramp and paddle across the Interlaken “ghost town” resort for a snack. The same place I took some MTB photos earlier in the week! I suffered from some geographic embarrassment, not realizing we were in the west basin not the east of twin lakes.

It ended up as a happy mistake (vale Bob Ross) landing on an isolate beach for a rest break. Not a soul in sight but a screech from the trees alerted us to a predator. We were treated to what I think was a kestrel or some other hawk or eagle. Magnificent against the backdrop of the high Rockies (and uncapturable on an iPhone or all but the most expensive camera gear so this one is captured in our memories). Realizing our (my) mistake we headed out to the channel between the two lakes (Twin Lakes.. it is what it says on the tin) and headed towards the old town of Interlaken, made up of cabins of the well-to-do who wanted to enjoy mountain life (history, it seems, repeats). Now home to bats and a welcome rest stop for hikers, bikers, and paddlers alike. There is something special to a place that cannot be reached by vehicle. Pardon my arrogance, but the filter to the lazy means only those that put in the effort can reach the special places and this keeps them quite and nice. We enjoyed a very pleasant lunch of salmon bagels, cliff bars and water and boarded our craft for the paddle back. It was close to 11am and, as predicted there were some impressive clouds developing on Mt Elbert. Only a matter of time until they have overcome mid-level dry layers and “gone deep” producing rain and, much to my worry, graupel inside the clouds and lightning. Also, as those clouds were rising, they were sucking in air along the Saguach range (of to which Mt Elbert belongs) creating winds across the lake. So we headed back with a cross tail wind and some fun waves (but no mirror surface).

2 Mile high brewing. Beer with altitude.

Back towards the channel with the full fetch of the wind across the east basin we had some small waves that were sometimes a hindrance and sometimes a fun benefit providing a bit of a surf. Through the channel the waves ceased to be part of the picture. Just as we approached the boat ramp the rain was starting on the southern shore of the Twin Lakes. We quickly hauled out and dropped out kayaks back in Twin Lakes. A highlight of the trip and as you, dear reader, can see from the photos, a spectacular, indelible memory for life. Arms sore, souls, again, very full we turned to fulfilling of the belly. And, boy, was the weather turning! Our luck was amazing (funny how clever folks are lucky). Back into Leadville and on the race across the Arkansas headwaters valley the atmosphere let off some fireworks with storms exploding all around us. With a couple of stops with Louise and River enduring the fact that, when the sky delivers, I am always on the clock we headed into Leadville for some chores and some great beer, food and company at 2-Mile-High Brewing. Don’t go looking for bowing and scraping service at this establishment. Beer was good. Not a fan of the larger, that need some work. All the ales were excellent, and food was great grub. Atmosphere was kind of the “Anti-Aspen”. I hope Leadville continues to value the down to earth nature of places like these. It attract folks like me who seek an authentic mountain experience.

Pure Mountains

On July 30, 2021 By scottcollisIn Cycling, Destinations, Hobbies, Travel, UncategorizedLeave a comment

Seriously, the best pic of a bike I have ever taken.

Today was a sampler of some of the best of the high parts of Colorado. Pure mountain biking, pure Colorado “highways” and Aspen, a pure (albeit touristy) mountain ski town. First, the mountain biking. Herein follows a review of the Turquoise (LINK) lake single track just out of Leadville. MTB project touts that the reviews and rating are submitted by local riders. I do not doubt this is the case for the blue rated out-and-back 6 mile (so 12 mile if you do it out-and-back) trail along the north shore of Turquoise lake but it shows a rider who considers themselves to be “blue” (intermediate) based out of Leadville are far more skilled than us flatlanders from the midwest.

The trail starts very easy, some rocks to navigate but plenty of space to prepare and recovery from the feature. I say this is mountain biking (specifically XC MTB) at its purest as, even though the trail is maintained (no erosion etc) zero attempt has been made to make the trail “flow”. Totally un-artificial and I rode it to see the amazing landscape unfold as it can only do on a loop. Some parts were a joy to ride, and some parts were technical, some other parts got gnarly where you did not have a clean run at a feature and you had rock gardens followed by rock gardens so you could never really get back on balance and some part were hike-a-bike for all but living legends. If you take at the MTB project link above you will see the peak in the middle, that is where most of the hike a bike is. The rest is ridable by any solid intermediate rider who is used to the trail and elevation. This intermediate rider is NOT used to the elevation and was riding it blind so more than a few flubs of a run up to a rock garden forcing a walk to get started. Louise will be happy to learn that there were no spills today. So, to answer the question: Was it worth it? Do you like technical rocky sections? This is for you.

Clouds forming over the ranges on either side of the Arkansas headwaters valley near Leadville.

Do you prefer flow but are willing to suck it up for world class views? It’s worth it but judge your risk tolerance carefully. Good news, the whole trail has cell service for your significant other to send the authorities your way, as always set a “no contact panic time”. Are you familiar with the world class views Leadville has to offer and love the flow and think pushing your bike is for chumps? I would say avoid. Let me put it this way: I chose not to do the out and back again, opting to take part of the Leadville 100 route and then turn off for a mad descent to the dam wall and where I parked. Strava file here: Lin k .

Arriving back at the cabin we discussed priorities. The weather was forecast to close in so we opted that this afternoon would be Aspen. We climbed the Independence Pass again (discovering it was named for the, now ghost, town of Independence. This time, instead of stopping and turning around we descended the other side. The road down on the Aspen side is very different. It starts a lot mellower than the Twin Lakes side but quickly narrows, a lot! The story on the Twin Lakes side was avalanche scars. On the Aspen side it is rock falls. This means the road narrows in several spots to a one lane road with some unwritten rule making it work. Crazy there are not more issues (perhaps there are but we witnessed none today). Aspen sits at around 7,900 feet, the air felt so thick! First impressions: Wow, parking is a challenge. Asking a local he directed us to a road on the edge of town which connects to Aspen’s wonderful walking trail system. The forecasted weather was arriving so we hot-footed it to our destination: Aspen brewing and tap. Great beer, amazing pizza, phenomenal WIFI and pretty steep prices. Yes, Aspen is not Leadville.

It caters to the well-to-do. However very nice for a treat. And it was great watching the storm rage outside over the mountains enjoying the best pizza I have had in ages. All that was left to do was wait out the storm, head back to the car and enjoy a spectacular drive over Independence pass watching the clouds roll off the pine-covered slopes back to the cabin. We asked and the mountains derived pure majesty today.

In the right place

On July 24, 2021 By scottcollisIn Cycling, Destinations, Hobbies, Travel, UncategorizedLeave a comment

Greetings from North Platte Nebraska! Today was the big day for us (keep in mind we don’t do big road trips). 7.5 hours across Iowa and Nebraska. We broke it down into four blocks with three rests. Iowa city to Des Moines. Des Moines to a rest stop. Rest stop to Kearney. And Kearney to North Platte. Kind of felt like we took a good luck potion as, well, we kept having good luck. First, the breakfast place we found in Des Moines; St Kilda café was awesome. Yes, it was Aussie themed, but it was not kitschy, it was good. Fantastic coffee and great food. I had salmon, poached eggs (amazing) and lentils. Rest stop was clean and has some lovely grounds for a quick walk to refresh the mind and legs. The arch at Kearney was 100% random, and very nice. Designed as a museum to Nebraska, Buffalo etc it had great amenities and River loved the gemstones they picked up there (highlight for them which was amazing given how random it was. For me, I had two highlights.

Our dinner spot, Pals brewing in North Platte and the 7+ crop dusters that entertained us along the plains. Oh and the changing landscape from hilly in Iowa to sandy and flat across Nebraska. Gives me a newfound awe for our chose home we will soon be citizens of. A very timely road trip! Ok, the crop dusters first. These planes were just amazing. I am an aviation geek. I used to forecast for Australia’s aviation center which sparked this interest. These planes were slow, low and agile. I swear they flew lower than the trucks driving along I80 and weaved between them. Like a Star Wars trench run! Pals brewing, good food, beer (REALLY GOOD BEER. And I know good beer) and super friendly service. Geographically speaking, if this was Australia there would be literally nothing. And in the middle of Nebraska there is not much. So it was amazing to find such quality in the corn fields outside of North Platte.

Great day traveling across the plains, really hits home the impact of national infrastructure and national security. It’s no longer about rifles and cannons, it is food security and highways. Seeing the wind farms in Iowa really tickled me! Energy from the wind, amazing engineering and farmers getting $$$$ for a small footprint on their land.

On a final note, I had an epiphany on the road. I did not realize how poor my mental health was. About 2 hours out of Iowa City I had a deep exhale as my eyes fixed on the distance, I realized I felt trapped in Chicago without knowing it. The vacation that seemed like a good idea was, in fact, a necessity. Our family likes distant horizons, it is what inspired us to uproot to a whole new country. Is the USA better than Australia? Not necessarily…

Is Australia better than the USA? Also, not necessarily… Are they different and better and worse in different ways. Absolutely. Right now, seeing the industrious plains of the Midwest it is driven home to me like better before that, for us, the USA is a land of massive opportunity and is the right place to be. Bring on the Mountains tomorrow!!!

North Platte Nebraska… Old, conservative and funky!

Go West

On July 23, 2021 By scottcollisIn Cycling, Destinations, Hobbies, Travel, Uncategorized1 Comment

Greetings readers. I have been slack. and not like salesforce slack.. Slack at updating this blog. News is we are undertaking our longest roadtrip ever in the USA. Actually, I thought it was the second longest as I drove with a friend from Gladstone QLD to Canberra way back in in the early 2000’s … But some googling tells me I was wrong and this will be my LONGEST roadtrip ever (yes I will tell you what it is soon). Difference is, as a stupid 20 year old I (with my friend… Can’t remember his name to save my life, straight shot it, 18 hours, 3 hour shifts, sleeping, running around the car… Good/bad times 🙂 )…

So, we are heading out on a long earned two week vacation to Colorado. Unlike the 20 year old me I now have responsibility for two other humans, Louise and River. So we are taking three days to trek across the great U.S.A. Staying in Iowa City, Iowa the first night, North Platte, Nebraska the second and, a place I have frequented often, Boulder Colorado (sorry friends there, no time to catch up on the way there, hopefully on the way home) on the third.

This post is being written from the lobby bar (YES Lobby bar! It’s been a long time thanks to COVID-19) from the Courtyard Marriott in Iowa City. Drive from Chicago could not have been smoother. Great stop Leclair, IA on the Mississippi river at a very nice German pub (AC was out and they were suffering bad from the “Big Quit”).

I’ll get to more details of our trip and plans later. In a nutshell: Mountains, stars, mountain biking and the kindness of strangers (Thank you thank you Bob Graham).. 90% vacation, 10% work on the road.

Fun note, call from WTTW Chicago to appear on Chicago Tonight as we went through Dekalb.. First time in over 10 appearances I turned them down.. Sorry folks.

We rolled into Iowa City, a college town, at around 4pm. Checked in and headed out on foot for grub and beer. Wow, so much development in the middle of the corn and soy of Iowa! Lovely town. Headed to a local, Sanctuary, on a very sweaty, Friday for a Pint.. Quaint… They headed by Vue, a rooftop bar at the new Hilton in the middle of town.. Vue had a view! Chatted with the staff and some very nice locals. Iowa city, thanks to the University is in the middle of a development boom, that’s why we are saving like mad for River’s college… We loved this place. Tomorrow is the big day. 7 hours, 24 minutes, to North Platte Nebraska. Lots more corn to come!

Software Defined Radio Reading of a Personal Weather Station

On July 16, 2020July 16, 2020 By scottcollisIn Coding, General Weather Geekery, Sage, Science Communications, UncategorizedLeave a comment

I am very lucky to be involved in the Sage Cyberinfrastructure project where I am kind of acting as “Chief Science Evangelist”. Basically I help motivate the hardware and software engineering with real world science use cases. Discussions with friends like Profs.

Pi4 with the RTL-SDR reading 915MHz data. Pi3 syncing via scp and displaying on UnicornHat-HD

Eric Brunning and Tim Logan have led me to Lightning detection as an interesting problem and several folks in the team are big fans of Software Defined Radios. So I purchased one (and then three more!) and started playing. Several months ago I purchased ($169) an Ambient W2902B personal weather station. It sends signals to a base station via PCM @ 915MHz. The base station then sends data to ambient you can download via an API. Issue is you can only download 1 minute data and the head unit transmits every 16s. I was ok with that for the price. Well, enter the software defined radio! I knew the RTL-SDR 2832U unit I have can tune into 915MHz.. And looking on gqrx I cold see the pulses so I thought I could write some code to decode the data..

Output from rtl_433. Note my Ambient 2903B tags itself as BOTH fineoffset-WH24 and 65B

Well cool thing is I did not have to! As I always say to my students: Google that! I did and found this excellent blog. It was as simple as a apt-get rtl-433 or, on the Raspberry Pi4 compiling from source and you have a pulse code decoder that can read everything from a weather station to a tire pressure gauge in your car. I then simply set it to run with a nohup on my Raspberry Pi4 and save JSONs to a file. I wrote some simple code in Python using ACT (a module that builds on xarray) to visualize.. You can see so much more in the 16s data! Like the turbulence reflected in dewpoint and temperature data when the boundary layer builds. Now I want to include a Software Defined Radio in every Sage node so we can connect instruments wirelessly using the 915 and 433MHz standards. More on lightning detection later!

Bingo! My own weather data saved and plotted without using a 3rd party API.

The weather in Chicago is like an angry teenager….

On April 15, 2020 By scottcollisIn General Weather Geekery, Science Communications, UncategorizedLeave a comment

As well stated by my friend Rajesh Sankaran “While I have you here, what is up with the weather? Why is it behaving like an angry confused teenager going through puberty here in Chicago/Midwest? This hot and cold is not fun.”. First, it’s not due to COVID19. Yes, as I said on WTTW Chicago Tonight, there are clear impacts on the atmosphere and our ability to predict it however, the variability that saw us go from shorts weather to snow is a normal seasonal phenomena. In previous posts I have compared the

upper level winds to “railway tracks” these winds are referred to at the Jet streams or jet stream pattern. At the moment these patterns are very buckled sending storm systems bearing tornadoes and hail into Alabama and southern states while pulling arctic air down over Chicago. If this were mid winter it would be a Chibera, however because things have been warming we are seeing snow and just above freezing at the surface… What is crazy about these snow systems is we are seeing snow at up to +5 degrees celsius! This is because the temperature is cooling so rapidly with height that snowflakes are not melting on their trip from the freezing level to the surface… Silver lining? well hopefully the snow is good for stay at home orders and social distancing.. meanwhile I am getting back into baking to make the house warm…

GFSNA_500_spd_000 — Current Jet Stream pattern that helps direct the movement of weather patterns…

OpenSky

Category: Uncategorized