Big Data Tools and Trends

FromData Skeptic

Start listening View podcast show

Big Data Tools and Trends

FromData Skeptic

ratings:

Length:

31 minutes

Released:

Feb 17, 2017

Format:

Podcast episode

Description

In this episode, I speak with Raghu Ramakrishnan, CTO for Data at Microsoft. We discuss services, tools, and developments in the big data sphere as well as the underlying needs that drove these innovations.

Released:

Feb 17, 2017

Format:

Podcast episode

Titles in the series (100)

Data Skeptic is a data science podcast exploring machine learning, statistics, artificial intelligence, and other data topics through short tutorials and interviews with domain experts.

Skip carousel

[MINI] Experimental Design: This episode loosely explores the topic of Experimental Design including hypothesis testing, the importance of statistical tests, and an everyday and business example. by Data Skeptic
16 min listen
[MINI] Decision Tree Learning: Linhda and Kyle talk about Decision Tree Learning in this miniepisode. Decision Tree Learning is the algorithmic process of trying to generate an optimal decision tree to properly classify or forecast some future unlabeled element based by... by Data Skeptic
13 min listen
Streetlight Outage and Crime Rate Analysis with Zach Seeskin: This episode features a discussion with statistics PhD student Zach Seeskin about a project he was involved in as part of the Eric and Wendy Schmidt Data Science for Social Good Summer Fellowship. The project involved exploring the... by Data Skeptic
33 min listen
[MINI] type i / type ii errors: In this first mini-episode of the Data Skeptic Podcast, we define and discuss type i and type ii errors (a.k.a. false positives and false negatives). by Data Skeptic
11 min listen
Introduction: The Data Skeptic Podcast features conversations with topics related to data science, statistics, machine learning, artificial intelligence and the like, all from the perspective of applying critical thinking and the scientific method to evaluate the... by Data Skeptic
4 min listen
[MINI] Ant Colony Optimization: In this week's mini episode, Linhda and Kyle discuss Ant Colony Optimization - a numerical / stochastic optimization technique which models its search after the process ants employ in using random walks to find a goal (food) and then leaving a... by Data Skeptic
15 min listen
[MINI] Cross Validation: This miniepisode discusses the technique called Cross Validation - a process by which one randomly divides up a dataset into numerous small partitions. Next, (typically) one is held out, and the rest are used to train some model. The hold out set can... by Data Skeptic
1 min listen
Mapping Reddit Topics with Randy Olson: My quest this week is noteworthy a.i. researcher Randy Olson who joins me to share his work creating the Reddit World Map - a visualization that illuminates clusters in the reddit community based on user behavior. Randy's blog... by Data Skeptic
30 min listen
Advertising Attribution with Nathan Janos: A conversation with Convertro's Nathan Janos about methodologies used to help advertisers understand the affect each of their marketing efforts (print, SEM, display, skywriting, etc.) contributes to their overall return. by Data Skeptic
76 min listen
[MINI] p-values: In this mini, we discuss p-values and their use in hypothesis testing, in the context of an hypothetical experiment on plant flowering, and end with a reference to the Particle Fever documentary and how statistical significance played a role. by Data Skeptic
17 min listen
Personalized Medicine with Niki Athanasiadou: In the second full length episode of the podcast, we discuss the current state of personalized medicine and the advancements in genetics that have made it possible. by Data Skeptic
57 min listen
Data in Healthcare IT with Shahid Shah: Our guest this week is Shahid Shah. Shahid is CEO at Netspective, and writes three blogs: Health Care Guy, Shahid Shah, and HitSphere - the Healthcare IT Supersite. During the program, Kyle recommended a talk from the 2014 MIT Sloan CIO Symposium... by Data Skeptic
57 min listen
The Right (big data) Tool for the Job with Jay Shankar: In this week's episode, we discuss applied solutions to big data problem with big data engineer Jay Shankar. The episode explores approaches and design philosophy to solving real world big data business problems, and the exploration of the wide... by Data Skeptic
50 min listen
Jackson Pollock Authentication Analysis with Kate Jones-Smith: Our guest this week is Hamilton physics professor Kate Jones-Smith who joins us to discuss the evidence for the claim that drip paintings of Jackson Pollock contain fractal patterns. This hypothesis originates in a paper by Taylor, Micolich,... by Data Skeptic
50 min listen
$Jackson Pollock Authentication Analysis with Kate Jones-Smith: Our guest this week is Hamilton physics professor Kate Jones-Smith who joins us to discuss the evidence for the claim that drip paintings of Jackson Pollock contain fractal patterns. This hypothesis originates in a paper by Taylor, Micolich,...$
[MINI] Bayesian Updating: In this minisode, we discuss Bayesian Updating - the process by which one can calculate the most likely hypothesis might be true given one's older / prior belief and all new evidence. by Data Skeptic
11 min listen
[MINI] Value of Information: A discussion about getting ready in the morning, negotiating a used car purchase, and selecting the best AirBnB place to stay at help frame a conversation about the decision theoretic principal known as the Value of Information equation. by Data Skeptic
14 min listen
[MINI] Noise!!: Our topic for this week is "noise" as in signal vs. noise. This is not a signal processing discussions, but rather a brief introduction to how the work noise is used to describe how much information in a dataset is useless (as opposed to... by Data Skeptic
16 min listen
[MINI] Selection Bias: A discussion about conducting US presidential election polls helps frame a converation about selection bias. by Data Skeptic
15 min listen
The Science of Online Data at Plenty of Fish with Thomas Levi: Can algorithms help you find love? Many happy couples successfully brought together via online dating websites show us that data science can help you find love. I'm joined this week by Thomas Levi, Senior Data Scientist at Plenty of Fish, to... by Data Skeptic
59 min listen
Data Science at ZestFinance with Marick Sinay: Marick Sinay from ZestFianance is our guest this weel. This episode explores how data science techniques are applied in the financial world, specifically in assessing credit worthiness. by Data Skeptic
31 min listen
[MINI] The Chi-Squared Test: The χ2 (Chi-Squared) test is a methodology for hypothesis testing. When one has categorical data, in the form of frequency counts or observations (e.g. Vegetarian, Pescetarian, and Omnivore), split into two or more categories (e.g. Male,... by Data Skeptic
18 min listen
[MINI] z-scores: This week's episode dicusses z-scores, also known as standard score. This score describes the distance (in standard deviations) that an observation is away from the mean of the population. A closely related top is the 68-95-99.7 rule which... by Data Skeptic
10 min listen
The Secret and the Global Consciousness Project with Alex Boklin: I'm joined this week by Alex Boklin to explore the topic of magical thinking especially in the context of Rhonda Byrne's "The Secret", and the similarities it bears to The Global Consciousness Project (GCP). The GCP puts forward the hypothesis that... by Data Skeptic
42 min listen
Guerilla Skepticism on Wikipedia with Susan Gerbic: Our guest this week is Susan Gerbic. Susan is a skeptical activist involved in many activities, the one we focus on most in this episode is Guerrilla Skepticism on Wikipedia, an organization working to improve the content and citations of... by Data Skeptic
70 min listen
Contest Announcement: The Data Skeptic Podcast is launching a contest- not one of chance, but one of skill. Listeners are encouraged to put their data science skills to good use, or if all else fails, guess! The contest works as follows. Below is some data about the... by Data Skeptic
12 min listen
[MINI] Partially Observable State Spaces: When dealing with dynamic systems that are potentially undergoing constant change, its helpful to describe what "state" they are in. In many applications the manner in which the state changes from one to another is not completely predictable,... by Data Skeptic
13 min listen
Easily Fooling Deep Neural Networks: My guest this week is Anh Nguyen, a PhD student at the University of Wyoming working in the Evolving AI lab. The episode discusses the paper Deep Neural Networks are Easily Fooled [pdf] by Anh Nguyen, Jason Yosinski, and Jeff Clune. It... by Data Skeptic
28 min listen
[MINI] Ordinary Least Squares Regression: This episode explores Ordinary Least Squares or OLS - a method for finding a good fit which describes a given dataset. by Data Skeptic
18 min listen
Economic Modeling and Prediction, Charitable Giving, and a Follow Up with Peter Backus: Economist Peter Backus joins me in this episode to discuss a few interesting topics. You may recall Linhda and I previously discussed his paper "The Girlfriend Equation" on a recent mini-episode. We start by touching base on this fun paper and get a... by Data Skeptic
24 min listen
Doubtful News, Geology, Investigating Paranormal Groups, and Thinking Scientifically with Sharon Hill: I had the change to speak with well known Sharon Hill (@idoubtit) for the first episode of 2015. We discuss a number of interesting topics including the contributions Doubtful News makes to getting scientific and skeptical... by Data Skeptic
31 min listen

More Episodes from Data Skeptic

Skip carousel

Modelling Evolution: Modeling evolutionary processes goes way beyond the Hardy-Weinberg Equilibrium we all learned in biology class. Natural selection comes from many sources like resources availability, mate preferences, competition. Modeling entire populations of... by Data Skeptic
41 min listen
Behavioral Genetics: It’s almost impossible to think about animal behavior without thinking of dogs! Our canine friends are a subspecies of wolf that has been co-evolving with us for tens of thousands of years. The transition from wolf to pet has required intense... by Data Skeptic
47 min listen
Signal in the Noise: In this episode, we are joined by Barbara Webb and Anna Hadjitofi. Barbara runs the Insect Robotics lab at the University of Edinburgh, and Anna is a PhD student at the School of Informatics at the university. She is interested in studying and... by Data Skeptic
42 min listen
Pose Tracking: Many researchers and students have painstakingly labeled precise details about the body positions of the creatures they study. Can AI be used for this labeling? Of course it can! Today's episode discusses Social LEAP Estimates Animal Poses (SLEAP), a... by Data Skeptic
51 min listen
Modeling Group Behavior: Our guest in this episode is Sebastien Motsch, an assistant professor at Arizona State University, working in the School of Mathematical and Statistical Science. He works on modeling self-organized biological systems to understand how complex patterns... by Data Skeptic
41 min listen
Advances in Data Loggers: Our guest in this episode is Ryan Hanscom. Ryan is a Ph.D. candidate in a joint doctoral evolution program at San Diego State University and the University of California, Riverside. He is a terrestrial ecologist with a focus on herpetology and... by Data Skeptic
36 min listen
What You Know About Intelligence is Wrong (fixed): We are joined by Hank Schlinger, a professor of psychology at California State University, Los Angeles. His research revolves around theoretical issues in psychology and behavioral analysis. Hank establishes that words have references and... by Data Skeptic
42 min listen
Animal Decision Making: On today’s episode, we are joined by Aimee Dunlap. Aimee is an assistant professor at the University of Missouri–St. Louis and the interim director at the Whitney R. Harris World Ecology Center. Aimee discussed how animals perceive information and... by Data Skeptic
37 min listen
Octopus Cognition: We are joined by Tamar Gutnick, a visiting professor at the University of Naples Federico II, Napoli, Italy. She studies the octopus nervous system and their behavior, focusing on cognition and learning behaviors. Tamar gave a background to the kind... by Data Skeptic
38 min listen
Optimal Foraging: Claire Hemmingway, an assistant professor in the Department of Psychology and Ecology and Evolutionary Biology at the University of Tennessee in Knoxville, is our guest today. Her research is on decision-making in animal cognition, focusing on... by Data Skeptic
38 min listen
Memory in Chess: On today’s show, we are joined by our co-host, Becky Hansis-O’Neil. Becky is a Ph.D. student at the University of Missouri, St Louis, where she studies bumblebees and tarantulas to understand their learning and cognitive work. She joins us... by Data Skeptic
49 min listen
OpenWorm: On this episode, we are joined by Stephen Larson, the CEO of MetaCell and an affiliate of the OpenWorm foundation. Stephen discussed what the Openworm project is about. They hope to use a digital C. elegans nematode (C. elegans for... by Data Skeptic
34 min listen
What the Antlion Knows: Our guest is Becky Hansis-O’Neil, a Ph.D. student at the University of Missouri, St Louis, and our co-host for the new "Animal Intelligence" season. Becky shares her background on how she got into the field of behavioral intelligence and biology. by Data Skeptic
42 min listen
AI Roundtable: Kyle is joined by friends and former guests Pramit Choudhary and Frank Bell to have an open discussion of the impacts LLMs and machine learning have had in the past year on industry, and where things may go in the current year. by Data Skeptic
51 min listen
Uncontrollable AI Risks: We are joined by Darren McKee, a Policy Advisor and the host of Reality Check — a critical thinking podcast. Darren gave a background about himself and how he got into the AI space. Darren shared his thoughts on AGI's achievements in the coming... by Data Skeptic
39 min listen
I LLM and You Can Too: It took a massive financial investment for the first large language models (LLMs) to be created. Did their corporate backers lock these tools away for all but the richest? No. They provided comodity priced API options for using... by Data Skeptic
24 min listen
Q&A with Kyle: We celebrate episode 1000000000 with some Q&A from host Kyle Polich. We boil this episode down to four key questions: 1) How do you find guests 2) What is Data Skeptic all about? 3) What is Kyle all about? 4) What are Kyle's thoughts on AGI?... by Data Skeptic
40 min listen
LLMs for Data Analysis: In this episode, we are joined by Amir Netz, a Technical Fellow at Microsoft and the CTO of Microsoft Fabric. He discusses how companies can use Microsoft's latest tools for business intelligence. Amir started by discussing how business intelligence... by Data Skeptic
29 min listen
AI Platforms: Our guest today is Eric Boyd, the Corporate Vice President of AI at Microsoft. Eric joins us to share how organizations can leverage AI for faster development. Eric shared the benefits of using natural language to build products. He discussed the... by Data Skeptic
34 min listen
Deploying LLMs: We are excited to be joined by Aaron Reich and Priyanka Shah. Aaron is the CTO at Avanade, while Priyanka leads their AI/IoT offering for the SEA Region. Priyanka is also the MVP for Microsoft AI. They join us to discuss how LLMs are deployed in... by Data Skeptic
35 min listen
A Survey Assessing Github Copilot: In this episode, we are joined by Jenny Liang, a PhD student at Carnegie Mellon University, where she studies the usability of code generation tools. She discusses her recent survey on the usability of AI programming assistants. Jenny discussed the... by Data Skeptic
26 min listen
Program Aided Language Models: We are joined by Aman Madaan and Shuyan Zhou. They are both PhD students at the Language Technology Institute at Carnegie Mellon University. They join us to discuss their latest published paper, PAL: Program-aided Language Models. Aman and Shuyan... by Data Skeptic
32 min listen
Which Programming Language is ChatGPT Best At: In this episode, we have Alessio Buscemi, a software engineer at Lifeware SA. Alessio was a post-doctoral researcher at the University of Luxembourg. He joins us to discuss his paper, A Comparative Study of Code Generation using ChatGPT 3.5 across 10... by Data Skeptic
40 min listen
GraphText: On the show today, we are joined by Jianan Zhao, a Computer Science student at Mila and the University of Montreal. His research focus is on graph databases and natural language processing. He joins us to discuss how to use graphs with LLMs... by Data Skeptic
31 min listen
arXiv Publication Patterns: Today, we are joined by Rajiv Movva, a PhD student in Computer Science at Cornell Tech University. His research interest lies in the intersection of responsible AI and computational social science. He joins to discuss the findings of this work that... by Data Skeptic
28 min listen
Do LLMs Make Ethical Choices: We are excited to be joined by Josh Albrecht, the CTO of Imbue. Imbue is a research company whose mission is to create AI agents that are more robust, safer, and easier to use. He joins us to share findings of his work; Despite "super-human"... by Data Skeptic
29 min listen
Emergent Deception in LLMs: On today’s show, we are joined by Thilo Hagendorff, a Research Group Leader of Ethics of Generative AI at the University of Stuttgart. He joins us to discuss his research, . Thilo discussed how machine psychology is useful in machine learning... by Data Skeptic
27 min listen
Agents with Theory of Mind Play Hanabi: Nieves Montes, a Ph.D. student at the Artificial Intelligence Research Institute in Barcelona, Spain, joins us. Her PhD research revolves around value-based reasoning in relation to norms. She shares her latest study, Combining theory of mind and... by Data Skeptic
38 min listen
LLMs for Evil: We are joined by Maximilian Mozes, a PhD student at the University College, London. His PhD research focuses on Natural Language Processing (NLP), particularly the intersection of adversarial machine learning and NLP. He joins us to discuss his latest... by Data Skeptic
26 min listen

Related podcast episodes

Skip carousel

Developer Experience and Data Protection with Skyflow’s Ram Muthukrishnan: In this episode, we delve into developer experience (DX) and its pivotal role in data protection, security, and privacy. Ram Muthukrishnan, a product manager at Skyflow, joins the show again to share insights into DX's definition, the key elements of... by Partially Redacted: Data Privacy, Security & Compliance
39 min listen
The Grand Vision And Present Reality of DataOps: A conversation about the grand vision and current realities of DataOps and how you can start on the journey toward more maintainable and reliable data systems. by Data Engineering Podcast
57 min listen
1628: The Future of Work - Data, Decentralization, and Automation.: Dr Mark van Rijmenam - The Digital Speaker by The Tech Talks Daily Podcast
31 min listen
Deciphering Data Architectures with James Serra by Insights Tomorrow
32 min listen
Realtime Data Applications Made Easier With Meroxa: Real-time capabilities have quickly become an expectation for consumers. The complexity of providing those capabilities is still high, however, making it more difficult for small teams to compete. Meroxa was created to enable teams of all sizes to deliver real-time data applications. In this episode DeVaris Brown discusses the types of applications that are possible when teams don't have to manage the complex infrastructure necessary to support continuous data flows. by Data Engineering Podcast
45 min listen
Credit Karma’s Ryan Graciano on Data Marts, Data Models, and Disrupting the Credit Landscape: Ryan Graciano, CTO of Credit Karma, discusses the importance of data hygiene, explainable AI and why data literacy is something you have to continuously work at. by The Data Chief
40 min listen
#76 Providing Financial Inclusion with Data science, with Vishnu V Ram, VP of Data Science and Engineering at Credit Karma by DataFramed
52 min listen
SingleStore CEO on High-Speed Database Currents by Tech Disruptors
47 min listen
Defining A Strategy For Your Data Products: The primary application of data has moved beyond analytics. With the broader audience comes the need to present data in a more approachable format. This has led to the broad adoption of data products being the delivery mechanism for information. In this episode Ranjith Raghunath shares his thoughts on how to build a strategy for the development, delivery, and evolution of data products. by Data Engineering Podcast
64 min listen
#92 Democratizing Data in Large Enterprises by DataFramed
43 min listen
2192: Couchbase CTO on Data Trends, Decentralisation, and Cloud Best Practice.: The brains behind Couchbase's leading data platform, CTO Ravi Mayuram, joins me on Tech Talks Daily to discuss how businesses can avoid the $8.75m cloud overspend revealed in the company's latest cloud evolution report. Ravi shares that even though... by The Tech Talks Daily Podcast
34 min listen
Dremio on Evolving Data-Technology Landscape by Tech Disruptors
44 min listen
The Beginning of Every Deep Learning Exercise With Manu Sharma and Brian Rieger: In today’s episode, we welcome Manu Sharma and Brian Rieger from Labelbox, a private company which we believe is leading training data solution for machine learning. We have had many conversations on this show about artificial intelligence from a hardw... by FYI - For Your Innovation
53 min listen
Datacenters - Think globally, act locally by The Intrazone by Microsoft 365
42 min listen
FoA 245: Agtech Product Strategy with Climate Corp Chief Product Officer Ranjeeta Singh: Today’s show connects back to episode 241 with Craig Rupp of Sabanto, where we talked about, among many other things, how the Climate Corp has been able to become a central data collection platform on so many large scale farms. Ranjeeta Singh,... by Future of Agriculture
30 min listen
Engineering for Data Privacy: Navigating Infrastructure, Security, and Compliance with Skyflow’s Roshmik Saha: In this episode Roshmik Saha, Head of Engineering at Skyflow, dives into the fascinating realm of data privacy and security solutions. Whether you're considering building your own privacy solution or seeking insights into the infrastructure requireme... by Partially Redacted: Data Privacy, Security & Compliance
45 min listen
Data Brew Season 2 Episode 9: Data Driven Software by Data Brew by Databricks
31 min listen
Designing Data Platforms For Fintech Companies: Working with financial data requires a high degree of rigor due to the numerous regulations and the risks involved in security breaches. In this episode Andrey Korchack, CTO of fintech startup Monite, discusses the complexities of designing and implementing a data platform in that sector. by Data Engineering Podcast
48 min listen
#213: Data Contracts: What They Are, Their Role, and Their Evolution with Shane Murray: When it comes to data, there are data consumers (analysts, builders and users of data products, and various other business stakeholders) and data producers (software engineers and various adjacent roles and systems). It's all too common for data... by The Analytics Power Hour
53 min listen
2283: How to Harness the Power of EDB Postgres: Marc Linster, the CTO of EnterpriseDB (EDB), joins me recently on Tech Talks Daily to talk about the benefits and growing popularity of open source database technology. With over 20 years of experience in the database industry, Marc is a leading... by The Tech Talks Daily Podcast
31 min listen
Microsoft Build: Application Security with Redis Cache (Shruti Pathak): Shruti Pathak, Senior Product Manager at Microsoft, talked about optimizing web applications by using Redis cache and things to consider to make it secure. Shruti also explained what in-memory databases are and how they improve performance. At the end we also talked about the program manager and her experience transitioning from a software development role. by The Women in Tech Show: A Technical Podcast
20 min listen
How Explainable AI is Critical to Building Responsible AI // Krishna Gade MLOps // Meetup #53 by MLOps.community
57 min listen
The Fourth Industrial Revolution Has Begun: Now’s The Time to Join: To prepare for a future dominated by data, says Lumen CTO Andrew Dugan, organizations need the right tools to collect, analyze, and act on it. by Business Lab
28 min listen
What Happened to Data Marts? by Insights Tomorrow
33 min listen
2503: LogicMonitor - Data Observability with Taggart Matthiesen: Today, I dive into the complex and often nebulous world of data observability with a leading expert in the field, Taggart Matthiesen, the Chief Product Officer at LogicMonitor. With an impressive career trajectory that includes pivotal roles at Lyft,... by The Tech Talks Daily Podcast
22 min listen
How Data Engineering Teams Power Machine Learning With Feature Platforms: Feature engineering is a crucial aspect of the machine learning workflow. To make that possible, there are a number of technical and procedural capabilities that must be in place first. In this episode Razi Raziuddin shares how data engineering teams can support the machine learning workflow through the development and support of systems that empower data scientists and ML engineers to build and maintain their own features. by Data Engineering Podcast
63 min listen
2827: Data, Decisions, and Dagster: Nick Schrock's Blueprint for Engineering Excellence: Nick Schrock, the innovative mind behind Dagster Labs and the renowned co-creator of GraphQL, joins me on Tech Talks Daily. Nick takes us through his illustrious journey from his foundational days at Facebook, where he spearheaded the Product... by The Tech Talks Daily Podcast
25 min listen
Data Labeling That You Can Feel Good About - Episode 89: An interview about the Cloud Factory platform for data labeling and social good in developing nations by Data Engineering Podcast
58 min listen
1174: Pepperdata - Removing the Blindfold to Control Cloud Spend: Ash Munshi, CEO of Pepperdata joins me on Tech Talks Daily to discuss the importance of removing the blindfold to control cloud spend. by The Tech Talks Daily Podcast
22 min listen
The ERP Minute Episode 57 - September 27, 2022 by The ERP Advisor
3 min listen

Discover this podcast and so much more

Big Data Tools and Trends

Big Data Tools and Trends

Description

Titles in the series (100)

More Episodes from Data Skeptic

Related podcast episodes