THE M5 CONFERENCE: ADVANCES IN FORECASTING - Machine Learning Methods Applied to Hierarchical Retailed Data 6-7 December 2021 Tribeca Rooftop
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
THE M5 CONFERENCE: ADVANCES IN FORECASTING Machine Learning Methods Applied to Hierarchical Retailed Data 6-7 December 2021 Tribeca Rooftop NYC Organizers:
Why Attend The M5 Competition
The M5 was the biggest and most ambitious of Aim
Why you must attend the M5 Who should attend the M5 all M Competitions to date. It ran from 2 March
Conference Conference? to 30 June 2020 and differed from the previous
The M5, the fifth expanded iteration of the highly-
regarded Makridakis (or M) Competitions, returned
The findings of the M5 Competition, one of the • Professionals working in companies or non-profit four ones in six important ways, some of which
in service of advancing the theory and practice of
largest forecasting competitions of all time, will be organizations in jobs that involve preparing were suggested by the discussants of the M4
Forecasting, in line with its enduring aim since the
presented and analysed at length over the course forecasts and estimating uncertainty Competition.
early 1980s.
of the two-day M5 Conference. In this respect, • Financial managers who prepare budgets and the • It used hierarchical sales data, generously made
the Conference will elaborate on the winning financial requirements for their firms available by Walmart, starting at the item level
In this respect, the aim of the M5 Competition was
methods, their practical applications and broader • Government officials requiring to predict receipts and aggregating to that of departments, product
similar to the previous four: that was to identify
implications, as well as how they can benefit and expenses categories and stores in three geographical areas
the most appropriate method(s) for different types
business and other organizations. • Hedge fund and other related managers who need of the US: California, Texas, and Wisconsin.
of situations requiring predictions and making
The M5 Conference Agenda includes distinguished to predict stock and other market variables • Besides the time series data, it also included
uncertainty estimates. Its ultimate purpose was
speakers from the major software/technology • Production managers requiring forecasts for their explanatory variables such as price, promotions,
to advance the theory of forecasting and improve
companies (Google, Kaggle, Walmart, Microsoft, production planning activities day of the week, and special events (e.g. Super
its utilization by businesses and non-profit
Amazon, Facebook, SAS and Target), as well as • Inventory managers who must predict the Bowl, Valentine’s Day, and Orthodox Easter)
organizations. Its other goal was to compare the
leading academics from top-tier universities. demand for a large number of items to figure out that affect sales which are used to improve
accuracy/uncertainty of ML and DL methods vis-à-
It features the presentation of the three most optimal inventory levels and reordering points forecasting accuracy.
vis those of standard statistical ones, and assess
accurate methods of the M5 Competition by the • Logistics and transportation managers whose • The distribution of uncertainty was assessed by
possible improvements versus the extra complexity
developers themselves who will also discuss how scheduling tasks need forecasts asking participants to provide information on four
and higher costs of using the various methods.
their methods can be implemented by others. • Academics teaching forecasting and related indicative prediction intervals and the median.
Their code will be available for free on GitHub. courses • The majority of the more than 42,840 time
The competition was a huge success with over
The Conference covers all critical aspects of • Students interested in forecasting for their courses series display intermittency (sporadic sales
100,000 entries from forecasters from over 100
forecasting, including combining methods and or for their research including zeros).
counties competing for the $100,000 in cash
introducing judgmental adjustments, paying • Instead of a single competition to estimate both the
prizes. The Accuracy Challenge currently stands as
special emphasis on the comparison of Machine point forecasts and the uncertainty distribution,
Kaggle’s 6th most popular competition of all time in
Learning and Statistical forecasting methods as there were two parallel tracks using the same
terms of the number of participating teams.
well as the assessment of uncertainty. Additionally, dataset, the first requiring 28 days ahead point
the Conference provides an excellent opportunity forecasts and the second 28 days ahead probabilistic
for networking and discussing the latest advances forecasts for the median and four prediction
in forecasting by world-renowned experts. intervals (50%, 67%, 95%, and 99%).
Finally, Nassim Nicholas Taleb will deliver a keynote • For the first time, it focused on series that display
address discussing uncertainty in forecasting and intermittency, i.e., sporadic demand including zeros.
Spyros Makridakis will present the major findings of
the M5 Competition and discuss how organizations
can benefit by such findings in order to improve
the accuracy of their predictions and assess their M5 Competition Prize Sponsors:
uncertainty more realistically.
Prof. Spyros Makridakis The Makridakis Open Forecasting Center
Spyros Makridakis is a pioneer in the field and one
of the foundational figures of Forecasting. He is
a Professor at the University of Nicosia, where he
serves as a Director of its Institute For the Future
(IFF), and is the Founder of the Makridakis Open
Forecasting Center (MOFC). In 1970 he joined
INSEAD as a Professor. He has authored, or co-
MOFC
authored, twenty-seven books and special issues
and more than 360 articles. His book “Forecasting Forecasts are essential for practically all business
Methods for Management, 5th ed. (Wiley)”, has decisions: from setting up appropriate inventory or
been translated in 12 languages and sold more service levels and credible budgets to evaluating
than 120,000 copies, while his book “Forecasting: long-term, strategic investments. The first objective
Methods and Applications, 3rd ed. (Wiley)”, has of the expanding field of forecasting is to offer
Learn. Forecast. Compete. Disseminate. Excel.
received more than 6,000 citations. Professor accurate predictions contributing to the success of
Makridakis was the Founding Editor-in-Chief of such decisions. Its second, equally important goal
the Journal of Forecasting and the International is to provide precise estimates of the uncertainty
Journal of Forecasting. inherent in all predictions and how to be able to
Professor Makridakis is the Organizer of the deal with the resulting risks.
renowned M (Makridakis) Competitions, the
premier academic forecasting competition in the The Makridakis Open Forecasting Center (MOFC)
world, which have fundamentally influenced the is one of the leading academic forecasting centers
theory and practice of forecasting for the past four in the world. It conducts cutting-edge research on
decades. His numerous papers and citations can be Forecasting, assisting organizations make accurate
found in his Google scholar profile: scholar.google. predictions, estimate levels of uncertainty, and
com/citations?user=hPpgXPMAAAAJ&hl=en rationally manage ensuing risks through training,
research, technology development, and consulting.
Ultimately, MOFC aims to expand the utilization
and value of Forecasting among business firms,
by identifying their needs, suggesting the most
appropriate way of fulfilling them, demonstrating
its benefits in reducing costs and/or improving
profits, while also avoiding untested practices.
M5 Competition
Award Winners & the Prizes
Prizes Winners
Prizes Description Amount Rank Competition Team Name Name
1A Most accurate point forecast $25,000 1 Accuracy YJ_STU YeonJun In
2A Second most accurate point forecast $10,000 Accuracy Matthias
2 Matthias Anderer
3A Third most accurate point forecast $5,000
3 Accuracy mf Yunho Jeon and Sihyeon Seong
4A Fourth most accurate point forecast $3,000
4 Accuracy monsaraida Masanori Miyahara
5A Fifth most accurate point forecast $2,000
5 Accuracy Alan Lahoud Alan Lahoud
6A Most accurate student point forecasts $5,000
Total: M5 Forecasting Competition – Point Forecasts $50,000
1 Uncertainty Everyday Low SPLices Russ Wolfinger and David Lander
2 Uncertainty GoodsForecast - Nick Mamonov Mamonov Nikolay
1B Most precise estimation of the uncertainty distribution $25,000
3 Uncertainty Ouranos Ioannis Nasios and Christos Iraklis Tsatsoulis
2B Second most precise estimation of the uncertainty distribution $10,000
3B Third most precise estimation of the uncertainty distribution $5,000 4 Uncertainty Marisaka Mozz Masakazu Mori
4B Fourth most precise estimation of the uncertainty distribution $3,000 5 Uncertainty IHiroaki Ihiroaki
5B Fifth most precise estimation of the uncertainty distribution $2,000 6 Uncertainty Ka Ho_STU Ka Ho Tsang
6B Most precise student estimation of the uncertainty distribution $5,000
Total: M5 Forecasting Competition – Uncertainty Distribution $50,000
Total: M5 Competition $100,000
M5 Conference Speakers
Spyros Makridakis is a Professor at the ed. (Wiley)”, has received more than 6,000 Anil Gaba received his Ph.D. in Decision He teaches courses on Uncertainty, Data, and
University of Nicosia, where he is a Director citations. Professor Makridakis was the Sciences at Duke University in the United Judgment (MBA), Probability and Statistics
of its Institute For the Future (IFF) and the Founding Editor-in-Chief of the Journal of States. He is currently the Orpar Chaired (PhD), and Bayesian Analysis (PhD). In
Founder of the Makridakis Open Forecasting Forecasting and the International Journal of Professor of Risk Management and Professor addition, he teaches modules on Judgments
Center (MOFC). In 1970 he joined INSEAD as a Forecasting and is the Organizer of the renown of Decision Sciences at INSEAD. He is also & Decision Making and Risk Management in
Professor. He has authored, or co-authored, M (Makridakis) competitions, that for the the Academic Director of Centre on Decision several executive development programmes
twenty-seven books and special issues and last 40 years have fundamentally influenced Making and Risk Analysis at INSEAD. He was all over the world including Europe, United
SPYROS MAKRIDAKIS more than 360 articles. His book “Forecasting the theory and practice of forecasting. His ANIL GABA Dean of Faculty INSEAD in 2006-2009, and States, China, Singapore, India, and the
University of Nicosia Methods for Management, 5th ed. (Wiley)”, numerous papers and citations can be found in INSEAD Dean of Faculty and Research Asia Campus in Middle East. He has won the Outstanding
has been translated in twelve languages and his Google scholar profile: scholar.google.com/ 2002-2006. Teacher Award INSEAD MBA Core Course
sold more than 120,000 copies while his book citations?user=hPpgXPMAAAAJ&hl=en His research is in the area of assessment and (Uncertainty, Data, and Judgment) fourteen
“Forecasting: Methods and Applications, 3rd use of subjective information, and analysis times. He also directs the International
of decisions under risk and uncertainty. His Directors Program (for Board of Directors) at
research has appeared in several academic INSEAD.
journals such as Management Science, He is a regular consultant and keynote
Nassim Nicholas Taleb spent more than 20 Taleb holds a PhD from the University of Paris
Operations Research, Marketing Science, speaker at various multinationals in areas
years as a derivatives trader before starting a and an MBA from the Wharton School. He is
and Journal of Risk and Uncertainty. He is a of judgments & decision making and risk
full-time career in research in the field of risk the author of the Incerto, a 4-volume essay
co-author (with S. Makridakis and R. Hogarth) management.
management and applied probability. on uncertainty (Antifragile, The Black Swan,
of the book, Dance with Chance: Making Luck
Taleb has been involved in risk-based policy Fooled by Randomness, The Bed of Procrustes),
Work For You.
making, advising the IMF, and the UK Prime and Dynamic Hedging (1997), a technical
Minister on model error and the detection clinical book on derivatives, in addition to
NASSIM NICHOLAS TALEB and mitigation of tail exposures. He has also Silent Risk, a freely available technical book
New York University been hired the RAND corporation and various (and reexpression of the Incerto) in applied Mr. Polemitis currently serves as the Chief was a member of the national committee
branches of the U.S government and has probability theory. Executive Officer of the University of Nicosia that designed the blockchain strategy for
testified twice for the United States Congress. and EDEX, as a Board member of EDEX and Cyprus.
UNICAF, and as a member of the Council of Mr. Polemitis is the managing partner of
the University of Nicosia. The University of Ledra Capital where he led early-stage
Nicosia (UNIC) serves over 14,000 students, investments in, or software development of,
Addison Howard is a Competitions Program edge technology to rise to the top through along with over 18,000 additional students Software-as-a-Service platforms in the areas
ANTONIS POLEMITIS in its affiliated academic institutions. UNIC of higher education, cryptocurrency, online
Manager at the Google Cloud AI group crowdsourced competition solutions.
University of Nicosia is the largest university in Cyprus and is video publishing and legal research. Mr.
Kaggle, the home of data science. By Addison has helped run dozens of
matching organizations stepping into the supervised machine learning competitions the largest English language university in Polemitis was previously a principal at ACG
world of Machine Learning for the first across a wide variety of disciplines. He southern Europe. Capital, a privately held multi-billion dollar
time, or by partnering with companies has Bachelor’s degrees in Mathematics, Mr. Polemitis helped found the world- investment firm and a partner based in
looking to deepen their roots in the data Economics, and Accounting, and a Master’s leading Digital Currency / Blockchain New York and London in the private equity
ADDISON HOWARD science community, Kaggle provides that degree in Accounting Initiative at the University of Nicosia, co- practice of Mercer Management Consulting
Google Cloud AI partnership and opportunity for cutting taught the first university cryptocurrency (now Oliver Wyman), one of the world’s
course in the world, and is regularly quoted leading strategy consultancies.
as an expert on cryptocurrency issues. He
Brian Seaman is a Senior Director of Data Brian has a demonstrated history of
Science at Walmart. While at Walmart, he working in the retail industry. He is a
has led data science teams across many strong research professional, skilled in
domains including sales forecasting, Data Science, Scalability, and Distributed
algorithmic pricing, financial analytics, Systems. He has a Bachelor’s degree in
marketing optimization, image processing Physics and mathematics from Rowan
and data extraction. University and a PhD in Physics from the
BRIAN SEAMAN University of Colorado Boulder.
Walmart
M5 Conference Speakers
Chris Fry is a Senior Staff Data Scientist at systems. Chris served as an Advisor on the Enno Siemsen is the Patrick A Thiele Organization Science, Journal of Operations
Google, where he leads data science efforts M5 Forecasting Competition, contributing Distinguished Chair in Business at the Management, Production & Operations
dedicated to driving efficient utilization of ideas to the design of the competition, Wisconsin School of Business, University of Management, Strategic Management Journal
Google’s datacenter infrastructure. He has and helping to bring the competition Wisconsin-Madison. He serves as the Associate and Manufacturing & Service Operations
worked in the field of forecasting for over onto Kaggle. He has a Bachelor’s degree Dean for MBA and Masters Programs, and Management. His work has also been featured
25 years, and has developed forecasts and in Operations Research and Industrial teaches courses in Sales & Operations in the Harvard Business Review and the
built forecasting systems for companies Engineering from Cornell University, and Planning as well as Project Management. His California Management Review. He is the
CHRIS FRY across a wide range of industries including a Master’s degree in Management Science ENNO SIEMSEN expertise is in the fields of predictive analytics, author of ‘Demand Forecasting for Managers’,
Google energy, pharmaceuticals, travel, apparel, and Engineering from Stanford. University of demand forecasting, sales & operations a book on forecasting and organizational
grocery retail, advertising, and computer Wisconsin-Madison planning, operations and supply chain decision making. He currently serves as a
strategy, and project management. department editor for the Decision Sciences
His research currently focuses on forecast Journal, and as an associate editor for
model selection, as well as on the use the Journal of Operations Management,
of augmented reality in manufacturing. Production & Operations Management and
Dan Goldstein is Senior Principal Researcher Max Planck Institute, where he was awarded
Siemsen has been published in leading the Journal of Supply Chain Management.”
at Microsoft Research in New York City. He the Otto Hahn Medal in 1997. He has
outlets such as Management Science,
was previously President of the Society for degrees in Computer Science and Cognitive
Judgment and Decision Making, a Professor Psychology and received his PhD in 1997 from
at London Business School and he taught the University of Chicago. George Athanasopoulos is a Professor, series with particular interest in forecasting
or researched at Yahoo Research, Wharton, Deputy Head and Director of Education at the and modelling tourism demand. He is
Columbia, Stanford, Harvard, and Germany’s Department of Econometrics and Business Associate Editor for the International Journal
DAN GOLDSTEIN Statistics at Monash University, Australia, of Forecasting and on the Editorial Board for
Microsoft
and is the President of the International the Journal of Travel Research. George has
Institute of Forecasters. He received his PhD co-authored with Professor Rob J. Hyndman,
in Econometrics from Monash University in Forecasting: Principles and Practice a free
Ellen Bonnell is a Principal Consultant Operating Officer for Candela. GEORGE 2007. A large body of his research has a strong online textbook, which is also available in
for TrendSavants. Ellen works with global Ellen has a Bachelor’s degree in Business ATHANASOPOULOS focus on forecasting large collections of time Korean and Chinese.
clients to improve software, methods and Economics from Indiana University Monash University
measurements for the sales, financial and Master’s degree in Supply Chain
and supply forecasts that drive Sales & Management from the University College
Operations Planning. Ellen specializes in Dublin, Michael Smurfit Graduate School of
Jim Hoover, Professor and Director of Artificial Doctorate of Business Administration from
highly accurate short-frequency forecasts Business. Ellen is currently a Board Member
Intelligence and Business Analytics Center, the University of Florida in 2017 and joined
ELLEN BONNELL that drive labor and inventory models. Ellen of the FORESIGHT Practitioner Advisory
TrendSavants University of Florida. He is the Chairman of the faculty there in 2019, where he is now
has also held positions as Head of Global Board.
the Foresight Advisory Board. Following a a Professor in the Marketing Department
Statistical Forecasting for Hilti and Chief
career in Operations Research for the U.S. focused on AI and Business Analytics. Jim
Navy, Jim became a Managing Director within initially served Foresight as Software Editor
Accenture Federal Services, with a focus and authored several articles on tracking
JIM HOOVER on supply-chain analytics. Jim received his forecast performance.
Evangelos Spiliotis is a Research Fellow at Forecasting Support Systems in 2017.
the Forecasting & Strategy Unit, National His research interests include time series University of Florida
Technical University of Athens (NTUA), forecasting, decision support systems,
where he also serves as Coordinator. He machine learning, and optimisation. He was
graduated from the School of Electrical part of the M4 and M5 teams, organising the Joannes Vermorel, based in France, is the engineering at the Ecole Normale Superieure
and Computer Engineering, NTUA (National respective forecasting competitions. Founder, majority shareholder and CEO (ENS Paris) for 6 years. Mr. Vermorel is an
Technical University of Athens) 2013, and at Lokad since February 2008. Lokad is alumnus of the Ecole Normale Superieure and
EVANGELOS SPILIOTIS got his PhD from the same School on a software company that specializes in an engineer of the Corps des Mines France.
National Technical University predictive supply chain optimization. Mr. He started his career at the AT&T Labs in New
of Athens Vermorel runs a series of live public supply Jersey, before pursuing a career in academia
chain lectures, emphasizing a modern in France which he interrupted to create his
JOANNES VERMOREL quantitative perspective. He taught software current company.
Lokad
M5 Conference Speakers
Jonathon Karelse, is the CEO of NorthFind heuristics in planning, organizations can make Michael Gilliland is Marketing Manager for of The Business Forecasting Deal (2010), and
Management. Jonathon Karelse is a global rapid improvements in real-world forecasting SAS forecasting software, and serves as Principal Editor of Business Forecasting:
leader in S&OP, demand planning and performance. He is a graduate of the MIT Sloan Treasurer on the IIF Board of Directors. Prior Practical Problems and Solutions (2015) and
operations. He is leading the conversation School of Management, a member of the 2019 to SAS, Mike held forecasting and supply chain Business Forecasting: The Emerging Role of
that Behavioral Economics represents Management Excellence cohort at Harvard positions in the food, consumer electronics, Artificial Intelligence and Machine Learning
the next big idea for decision making and Business School, and is a member of the Harvard and apparel industries. He is Associate Editor (2021).
planning in most companies. He believes that Business Review’s Advisory Council. of IIF’s practitioner journal Foresight, author
JONATHON KARELSE by understanding the causes of biases and MICHAEL GILLILAND
NorthFind Management SAS
Keith Ord is a Professor Emeritus at the He is author or co-author of 11 books including
Paul Goodwin is Emeritus Professor of Until 2015 he was an Editor of the International
McDonough School of Business, Georgetown Forecasting with Exponential Smoothing and of
Management Science at the University of Bath, Journal of Forecasting and, since 2007, he
University. His research interests include the textbook Principles of Business Forecasting.
UK. He has a PhD from Lancaster University has written the Hot New Research column in
forecasting using state space methods, epidemic He has published over 100 research papers in
and his research has focussed on methods for Foresight: The International Journal of Applied
forecasting epidemics, spatial statistics and any forecasting and other journals.
incorporating management judgment into Forecasting. His books include Forewarned:
applied statistics problems that come along.
forecasts to improve accuracy. He has provided A Sceptic’s Guide to Prediction (Biteback
forecasting advice to many organizations and Publishers) and Profit from your Forecasting
KEITH ORD
PAUL GOODWIN in 2013 he was elected as an Honorary Fellow Software (Wiley).
Georgetown University University of Bath of the International Institute of Forecasters.
Mahdi Yousefi received his PhD in Electrical he spent more than one and a half year as the
Engineering from Texas A&M in 2013, focusing science/tech lead of the Walmart eCommerce Pierre Pinson is a Professor of Operations in Meteorology, Power Systems Engineering,
on optimal and stochastic control, systems inventory optimization team and later the Research at the Technical University of Denmark Statistics and Operations Research. He has been
biology and cancer genomics. He has done demand forecasting team. He joined Target in (DTU, Dept. of Technology, Management and a visiting Researcher at the University of Oxford,
extensive research on reproducibility issues in 2019 and has been with its enterprise demand Economics). He is an IEEE Fellow and an ISI/ the University of Washington in Seattle, the
small-sample classification, clustering and error forecasting team since then. His time-series Clarivate highly-cited researcher (2019 & 2020). European Center for Medium-range Weather
estimation problems, which are common in forecasting algorithm was ranked 4th in the NN3 He is the Editor-in-Chief of the International Forecasts (ECMWF, UK), a visiting Professor at
MAHDI YOUSEFI medical sciences. After his PhD, he spent four 2006/07 Forecasting Competition for Neural Journal of Forecasting. His main focus areas École Normale Supérieure (Rennes, France) and
Target years at The Ohio State University as a Research Networks & Computational Intelligence. PIERRE PINSON cover forecasting, optimization and game a Simons fellow at the Isaac Newton Institute
Professor. He joined WalmartLabs in 2017, where Technical University of theory, with application within energy, (Cambridge, UK).”
Denmark logistics. He has published in leading journals
Maria P. Michailidis is a Professor at the editor of numerous journal articles and
Management and MIS department and the book chapters. In addition, she possesses Robert L. Winkler is James B. Duke Professor in the Frank P. Ramsey Medal for significant
Director of the Markidakis Open Forecasting a considerable industrial and teaching the Fuqua School of Business and Professor in contributions to decision analysis. Recent work
Center (MOFC) of the Institute For the experience from work she has done in the the Department of Statistical Science at Duke involves probability forecasting, combining
Future (IFF), of the University of Nicosia, United States and Cyprus. Her research University. His primary research areas include forecasts, decision modeling, stochastic
Cyprus. She has served as the Dean of the interests include topics related to industrial decision analysis, Bayesian statistics, forecasting, dominance, sequential decision making, and
School of Business from 2012-2017 and organizational psychology and her work has and risk analysis, and he has published multiattribute utility.
MARIA MICHAILIDIS Head of the Management and MIS from been supported by European grants, including extensively in these areas. He was awarded
University of Nicosia 2008-2012. She has been educated in the projects such as Leonardo da Vinci, Socrates/ ROBERT L. WINKLER
USA where she obtained her Doctorate Erasmus Thematic Networks and Grundtvig Duke University
from the University of Massachusetts. She Multilateral Projects and COST Actions.
specializes in Management, Social and Clinical She is a member of the Board of Directors
Psychology, Educational Leadership. She of the Elpida Foundation for children with
is Psychotherapist in the Person-Centered cancer and leukemia and of the A.G. Leventis
Approach (PCA). She is the author and Foundation Scholars Association.M5 Conference Speakers
Sihyeon Seong is a CEO and AI researcher of mofl Yeon Jun In is an undergraduate student in the
Inc. His primary research area is the optimization Department of Environmental Engineering and
of extremely high-dimensional spaces. He the Department of Industrial and Management
received his Ph.D. in Electrical Engineering from Systems Engineering at Kyung Hee University
the Korea Advanced Institute of Science and (KHU). He has a patent on time series demand
Technology (KAIST) in 2019 and B.S. Electrical forecasting. He is an enthusiastic AI researcher,
Engineering and Bio and Brain Engineering from and Kaggler. His research interests include
SIHYEON SEONG KAIST. YEON JUN IN machine learning, deep learning, and time series
mofl Inc Kyung Hee University data analysis in Industrial domain.
Slawek Smyl is a Data Scientist specializing in Intelligence in Forecasting International Time Yunho Jeon is a CTO and AI researcher of mofl Engineering from the Korea Advanced
forecasting, currently with Facebook. He holds Series Competition 2016, got a third place in Inc. He received a B.S. degree in mathematics Institute of Science and Technology (KAIST)
Master’s degree in Physics from Jagiellonian the Global Energy Forecasting Competition and an M.S. degree in computer science from in 2019. He also worked at SK telecom, Korea,
University, Poland, and the M.Eng. degree in in 2017, and won the M4 Forecasting Seoul National University in 2006 and 2008, as a Research Scientist in 2019. His research
Information Technology from RMIT University, Competition in 2018. respectively. From 2008 to 2019, he worked at interests include computer vision, deep
Australia. He won the Computational the Agency for Defense Development, Korea. learning, and time series forecasting.
He received his Ph.D. degree in Electrical
SLAWEK SMYL YUNHO JEON
Facebook mofl Inc
Tim Januschowski is a Machine Learning in the context of DevOps and business metrics.
Science Manager in Amazon AI Labs. He Tim’s personal interests in forecasting span
has worked on forecasting since starting applications, system, algorithm and modeling
his professional career. At Amazon, he has aspects and the downstream mathematical
produced end-to-end solutions for a wide programming problems. He studied
variety of time series analysis problems. In Mathematics at TU Berlin, IMPA, Rio de Janeiro,
forecasting, from demand forecasting to server and Zuse-Institute Berlin and holds a PhD from
TIM JANUSCHOWSKI capacity forecasting and in anomaly detection University College Cork.
Amazon AI Labs
Yael Grushka-Cockayne is an Altec Styslinger and behavioral decision-making. While
Foundation Bicentennial Professor of Business visiting Harvard Business School, Yael taught
Administration, a Senior Associate Dean for Technology and Operations Management
Professional Degree Programs, at Darden and Applied Business Analytics. Yael worked
School of Business at the University of Virginia. in San Francisco as a Marketing Director of
She teaches courses on Decision Analysis, an ERP company and as a Consultant to
Project Management, and Data Science in international firms. Yael is an Associate Editor
YAEL GRUSHKA- Business. Her research and teaching activities at Management Science, Operation Research,
COCKAYNE focus on decision analysis, data science, and Decision Analysis. She has a Bachelor’s
University of Virginia business analytics, forecasting, forecast degree from Ben-Gurion University; Master’s,
aggregation and the wisdom of crowds, from the London School of Economics and a
decision analysis, project management, Ph.D., MRes, from the London Business School.M5 Conference Agenda
Day 1 - Monday, 6 December Day 2 - Tuesday 7 December
8:30am - 9:30am Breakfast 8:00am - 9:00am Breakfast
9:30am - 10:00am Welcome / Opening Remarks 9:00am - 9:10am Opening Remarks
• Prof. Maria Michailidis, Director of the Makridakis Open Forecasting Prof. Maria Michailidis, Director of the Makridakis Open Forecasting
Center (MOFC), University of Nicosia Center (MOFC), University of Nicosia
• Mr. Antonis Polemitis, CEO of the University of Nicosia
• Prof. Anil Gaba, The ORPAR chaired Professor of Risk Management 9:10am - 9:40am Will Artificial Intelligence Take Your Job?
Academic Director, Centre for Decision Making and Risk Analysis, INSEAD Addison Howard, Google Cloud AI
• Prof. Spyros Makridakis, Founder of the Makridakis Open Forecasting Center,
MOFC & Director of the Institute For the Future, University of Nicosia 9:40am - 10:10am Keynote Address
The Changing Role of Forecasting in the Age of Cloud Computing
10:00am - 10:30am Keynote Address Chris Fry, Google
Forecasting and the COVID-19 Pandemic
Nassim Nicholas Taleb, New York University 10:10am - 10:40am Coffee Break and Networking
10:30am - 11:00am The Role of Judgment in Forecasting 10:40am - 11:30am The Most Accurate M5 Accuracy Methods
Dan Goldstein, Microsoft Summary of Findings
• Enno Siemsen, University of Wisconsin-Madison
11:00am - 11:30am Coffee Break and Networking • Yeon Jun In, Kyung Hee University, South Korea (KHU),
Winner of the Accuracy challenge, M5 Competition
11:30am - 12:00pm A Collaborative and Market-based View of the Future of Forecasting • Sihyeon Seong & Yunho Jeon, MOFL Inc., South Korea,
Pierre Pinson, Technical University of Denmark & Editor-in Chief IJF Third winner of the Accuracy challenge, M5 Competition
12:00pm - 12:30pm My New Forecasting Projects at Facebook 11:30am - 3:00pm The Value and Applicability of the M5 Competition
Slawek Smyl, Facebook Chairperson Jonathon Karelse, NorthFind Management
12:30pm - 2:00pm Buffet Lunch and Networking 11:30am - 12:00pm Applicability of the M5 to Forecasting at Walmart
Brian Seaman, Walmart
2:00pm - 3:40pm The Uncertainty Challenge
Chairperson Anil Gaba, INSEAD 12:00pm - 12:30pm Forecasts that Users Trust: Lessons Learned from a Large Scale
Demand Forecasting Project
2:00pm - 2:50pm Assessment of Uncertainty for Decision-Making Mahdi Yousefi, Target
Anil Gaba, INSEAD | Robert L. Winkler, Duke University
12:30pm - 2:00pm Buffet Lunch and Networking
2:50pm - 3:40pm The Most Precise Uncertainty M5 Methods. Summary of Findings
Evangelos Spiliotis, National Technical University of Athens (NTUA) 2:00pm - 2:30pm Deep Probabilistic Forecasting
Tim Januschowski, Amazon AI Labs
The Role of Uncertainty in Supply Chain: Lessons from the M5 Competition
Joannes Vermorel, Lokad 2:30pm - 3:00pm M5 Competition: How Organizations can Benefit from its Findings
Mike Gilliland, SAS
3:40pm - 4:00pm Coffee Break and Networking
3:00pm - 3:30pm The Past and Future of Forecasting Competitions and the
4:00pm - 5:45pm Theory and Practice of Forecasting: Future Challenges Forthcoming M6 Competition
Chairperson Antonis Polemitis, University of Nicosia Prof. Spyros Makridakis, Founder of the Makridakis Open Forecasting
Center, MOFC & Director of the Institute For the Future, University of
4:00pm - 4:30pm The Machine Learning (ML) is Better – When? Nicosia
Keith Ord, Georgetown University
4:30pm - 5:00pm Judgmental Forecasts and Judgmental Adjustments to Statistical/ML Forecasts
Paul Goodwin, Universty of Bath
5:00pm - 5:45pm Panel Discussion: Expanding the Use of Systematic Forecasting
in Organizations and Improving its Value: The UFO Challenge
• George Athanasopoulos, Monash University and President of IIF
• Ellen Bonnell, TrendSavants
• Yael Grushka-Cockayne, University of Virginia
• Jim Hoover, University of Florida
5:45pm - 6:50pm Free time
7:00pm - 9:00pm Gala Dinner & Award Ceremony of the M4 Competition PrizesM5 Conference Venue Register NOW
Tribeca Rooftop
One of New York City’s premier event venues,
Tribeca Rooftop has a well-earned reputation
for outstanding culinary expertise, impeccable
service, and warm hospitality. This chic industrial
event venue is located in the trendy downtown
neighborhood of Tribeca, New York City. It is
located on a cobblestone street in a 1920’s
Ticket prices
printing press building. The 25ft glass atrium and
large windowed walls offer a magnificent view of
Manhattan as well as the Hudson River. Normal $1450
The Tribeca Rooftop played host to the two- Academic $875
day M4 Conference, in 2018, assembling 20+
distinguished speakers from high-tech giants
Student $450
and software firms, including Google, Microsoft,
Amazon, Uber and SAS, as well as leading
academics from top-tier universities.
Address:
2 Desbrosses Street
New York, NY 10013
Closest Parking Garage:
Manhattan Parking Group
34 Desbrosses St.
(between Washington & West Sts.)
Contact:
(+1) 212.490.3460 mofc.unic.ac.cy/m5-conference-overview
The registration price includes the Breakfasts, Lunches,
AM/PM Coffee Breaks for Day 1 & Day 2, the Gala Dinner
inquiries@tribecarooftopnyc.com
on Day 1 and the Conference Material
Venue info can be found here:
www.tribecarooftopnyc.com
mofc.unic.ac.cy/m5-conference-overviewMakridakis Open Forecasting Center (MOFC)
2021 Membership Opportunities
MOFC Partner MOFC Sponsor MOFC Supporter
$60,000 (5 maximum) $25,000 $10,000
5 seats available for MOFC
Seat on MOFC Governing Board
partners
Corporate advisory services 3 Days (Director level) 3 Days (non-Director level)
Complimentary Enrolments to Applied Forecasting Course 10 6 2
Complimentary M conference Tickets 10 6 2
Advert in M conference brochure Full Spread Full page Half page
Logo on MOFC/conference website and all conference marketing materials
Sponsors desk at the back of the room at the M Conference
(plus 2 complimentary tickets for staff)
M Conference Coffee break Breakfast Lunch Dinner
Catering Sponsorship sponsorship sponsorship sponsorship sponsorship
$7,650 $15,000 $27,000 $30,000
Opportunities
x3 coffee breaks x2 breakfasts x2 lunches x1 dinner
for 150 people @ $17.00 for 150 people @ $50.00 for 150 people @ $90.00 for 150 people @ $200.00
AM & PM on Day1 & AM on Day2 Breakfasts on Day1 and Day2 Lunches on Day1 and Day2 Dinner at the end of Day1
* All MOFC membership contributions or donations will be used to support the Centre’s operation
(research, education, M competition & conference) to improve
our understanding and use of forecasting in organizationsAnnouncing the M6 Competition
The M6 Financial Forecasting Competition will be
The M6 US Stocks and International Assets organized and run in a similar way to the previous
Financial Forecasting Competition five competitions, attracting data scientists,
statisticians, financial experts/analysts, economists,
and related specialties from around the world,
A March 11, 2021, article in the WSJ entitled “Stock The M6 competition is a duathlon consisting of contending to win the substantial prizes being
Pickers Trailed Market Again in Roller Coaster 2020: two related challenges, involving two types of offered. M6 will be live, lasting a whole year, and
Some 60% of U.S. large-cap stock-picking funds investments, both stocks and asset classes. The open with all information about the competition
lagged behind S&P 500 in 2020, marking 11th first category comprises 50 large capitalization US and its results made available to anyone with
straight year of underperformance” exemplified stocks and the second encompasses 50 broadly an interest in the results. Moreover, once the
the well-known finding that active, professional traded international ETFs (assets) covering competition is finished its findings, together with
investment managers do not beat, on average, international stocks, bonds, commodities, and background material and commentaries will be
random stock selections. On the other hand, currency markets. Competition participants published in a special issue of the International
legendary investors like Warren Buffett, Peter Lynch would need to complete three tasks. First, rank Journal of Forecasting (IJF) with the goal of
and George Soros as well as celebrated firms as the attractiveness of all 50 stocks and 50 ETFs on a learning as much as possible about financial
Bridgewater Associates, Renaissance Technologies, scale from 1 to 5 and specify their level of certainty forecasting and the related factors driving
DE Shaw and several others have achieved about such rank, second, choose the stocks/assets investment returns and how investors can improve
phenomenal results, amassing returns impossible and the percentage they would like to invest in the accuracy of their forecasts and mitigate the
to justify by mere chance. It is the express purpose each and third provide information on how their uncertainty associated with them in order to build
of the M6 competition to investigate this paradox by selection was made. Consequently, they will be robust portfolios and achieving consistent, above
shedding as much light as possible to the following evaluated for (a) their forecasting performance, (b) average returns.
seven assertions: their return on investment and (c) their combined
The M6 competition will be live, lasting for twelve
achievement in (a) and (b), with prizes awarded to
1. The relationship between forecasting accuracy months, starting in February 2022, and ending a
each of the three categories.
and investment returns. year later in 2023. It will consist of a single month
2. The relationship between the perceived trial run and 24 rolling origins for participants to
uncertainty around such accuracy and returns. provide their submissions and be evaluated when
3. The contribution of judgment versus that of the actual data becomes available. The 24 rolling
objective information in selecting stocks and origins (4 weeks, repeated for six months) will
assets. be in alternate months, leaving the intervening
periods to participants to evaluate the submitted
4. How forecasting accuracy and uncertainty are
results, assess their performance and if necessary,
assessed in making investment decisions.
modify their strategy for the remaining period of
5. The influence of judgmental biases in the way
the competition. The schedule of the competition
investment decisions are made.
as well as detailed guidelines for participation is
6. The importance of a consistent investment available in an accompanying document that will
strategy. be uploaded to the M6 website.
7. The role of luck in achieving consistent, above
average returns.M6 Competition
Announcing the M6 Competition Sponsorship Opportunities
The M6 competition introduces the following ● Running a duathlon and asking for investment
innovations over the previous five ones: choices in stocks and financial assets,
● Being live instead of concealing part of the data substantially enlarges the scope of the
to evaluate performance, will allow participants competition and what can be investigated and
to: learned, including among others:
• Search for and use any available • The relative forecasting performance
information that could improve forecasting between stocks universe versus the asset
performance and returns on investments. class universe
• Incorporate, in addition to numerical • Differences in the Investment opportunity
inputs, judgmental ones about the set (risk/return tradeoff) between of Number
economy, the industry, and the firms different classes of financial assets Package Price of packages
that participants would invest to improve • The connection between forecasting available
forecasting performance and returns on accuracy and investment returns, split by
investments. asset class Platinum $100K 1
● In addition to measuring forecast performance, • The connection between the correct
we capture decisions made based on these appreciation of uncertainty and investment
Gold $50K 2
forecasts returns
• We can therefore measure the extent to Diamond $30K 3
• The role of judgment in forecasting
which portfolio construction decisions performance by type of prediction
detract investment value, given the value of Silver $20K 5
• The role of judgment in investment returns
forecast performance
by type of investment
● Having 24 rolling origins to evaluate
Bronze $10K
• The role of luck in achieving above average
performance (instead of a single one) allows
returns
participants to learn from experience, improve
• The influence of fat tails in the evaluation
their methodology and potentially achieve
of forecasting performance and investment
higher returns.
returns
Detailed information about the M6 competition and complete guidelines for participating
will be announced shortly.mofc.unic.ac.cy/m5-conference-overview
You can also read
