Chapter 1 - Getting Started

1.01 Acquiring a License (16 min.) Sample Lesson
1.02 Reference & Course Materials (9 min.)
1.03 Installing TIBCO Cloud Spotfire (4 min.)
1.04 Data & Row Types (25 min.)
1.05 Spotfire Bug (3 min.)
1.06 Exercise 1 (18 min.)
1.07 Spotfire Interface Cleanup (2 min.)

Chapter 2 - Spotfire Interface

2.01 Spotfire Interface (15 min.)
2.02 Hidden Panes (7 min.)
2.03 Data Canvas (5 min.)
2.04 Exercise 2 (10 min.)

Chapter 3 - Combining Data Tables

3.01 Joins & Unions (16 min.)
3.02 Exercise 3 (38 min.)

Chapter 4 - Visualizations

4.01 Visualization Types - Part 1 (14 min.)
4.02 Visualization Types - Part 2 (18 min.)
4.03 Visualization Types - Part 3 (11 min.)
4.04 Exercise 4 - Scatter Plots - Part 1 (33 min.)
4.05 Exercise 4 - Scatter Plots - Part 2 (23 min.)
4.06 Exercise 5 - Line Charts (16 min.)
4.07 Exercise 6 - Bar Charts (29 min.)
4.08 Exercise 7 - 3D Wellbore Diagram (15 min.)

Chapter 5 - Calculated Columns

5.00 - Expression Syntax (11 min.)
5.01 - Exercise 8 - Add a Calculated Column (19 min.)
5.02 - Exercise 9 - Case Statement (12 min.)
5.03 - Exercise 10 - OVER Functions (18 min.)
5.04 - Exercise 11 - Pseudo IP90 (33 min.)
5.05 - Exercise 12 - Map Chart (37 min.)
5.06 - Exercise 13 - Navigation (16 min.)
5.07 - Exercise 14 - Well Sticks (8 min.)
5.08 - Exercise 15 - Wellbore Traces (Lines) (11 min.)
5.09 - Exercise 16 - Wellbore Traces (Polygons) (11 min.)
5.10 - Exercise 17 - Feature Layer (10 min.)
5.11 - Exercise 18 - WMS Map Layer (7 min.)
5.12 - Exercise 19 - Image Layer (7 min.)
5.13 - Exercise 20 - Map Export (7 min.)
5.14 - Exercise 21 - Exporting Data & Visualizations (11 min.)

Chapter 6 - Data Limiting

6.01 Data Limiting (10 min.)
6.02 Data Limiting Exercise (13 min.)
6.03 Limit by Filters (18 min.)
6.04 Limit by Filters Exercise (18 min.)
6.05 Limit by Marking (4 min.)
6.06 Limit by Marking Exercise (7 min.)

Spotfire® for Oil & Gas Professionals - Beginner / Chapter 1 - Getting Started

Lesson 1.01 Acquiring a License

Access All SAGA Wisdom Training content Subscribe

Already a member? Sign in

Access All SAGA Wisdom Training content Subscribe

Already a member? Sign in

Transcript

01. Lesson 1.01: Acquiring a License 02. Acquiring a License 03. Acquiring a Professional License 04. Acquiring a Professional License (cont.)05. Acquiring an Academic License

01. Lesson 1.01: Acquiring a License

Everybody, my name's Bryan McDowell. I'll be the one teaching you on the Spotfire course. I'll give you a little background about myself. I did my bachelors in geology at Texas A&M, a Master's of Petroleum Engineering at Colorado School of Mines and I'm currently finishing out my Ph.D. at Mines and Geology. I've been using Spotfire for about 10 years, everything from new ventures exploration to actually just ETL type work, prepping it for a SQL, and then all the way to doing a full blown reservoir engineering dashboards and wells production dashboard from one man shops all the way up to billion dollar companies.
In this course, we're going to go through using TIBCO Spotfire for oil and gas applications. So throughout the course, we're going to learn how to #1 - bring in data tables, #2 - how to do calculations, do some data transformations, plot production data. And at the end, we're actually going to give you all some freebies, as well as preparing data to come in, whether it's cleaning the tables up and so on and so forth, as well as going through what I think are some pretty nifty well spacing calculations and also some Python functions. So we're excited to have you all here and with that, let's get started.
OK, so everybody, the first section we're going to go through is actually just getting started a Spotfire. So a lot of other software, the hardest part is actually just downloading the data and figure out which license you want. This is definitely an issue when we talk about Spotfire as well, when we're talking about whether you want to run this on-prem, so on your own servers vs. on the cloud. A lot of it has to do with the size of shop you are and what you choose is really going to drive what functionality you're going to be able to use.

02. Acquiring a License

So the first thing we need to look at is actually acquiring a professional license.

03. Acquiring a Professional License

So if you actually go out to TIBCO's website, so I've got the little hyperlink here that you can go look at the pricing or really it's the additions, you have to contact them for pricing. They've got 4 different options out here that you can run through. The big 2 that you really care about are going to be Cloud Spotfire and the Spotfire Platform. Now, Cloud Spotfire means that although it says cloud, you can run it on your desktop. The naming here is really is that is running off of cloud servers, you run off TIBCO's servers rather than your own. The flipside of this is Spotfire Platform which used to be called on-prem. So the on-prem version means effectively you're running it on your own servers, whether that's in-house like i.e. your servers are located at your office, or whether that's actually on the cloud.

04. Acquiring a Professional License (cont.)

So if you look at these in a little bit more detail, the TIBCO Cloud Spotfire is really the best solution, I would say, for small teams. So probably less than 10 people. You could get away with it with more folks, kind of depending on how much you're actually overlapping from the different courses. So the pros of this one: #1 is it runs off of TIBCO servers. So if you're like me in a very small shop, I do not have the talent or the time to actually stand up servers on my own. So here we're actually relying on TIBCO servers which are actually hosted out on Amazon Web Services (AWS) to actually run the database and the actual kind of application in the background. The good thing about this is 1) is that you don't need to purchase those servers, OK. So that saves you along the cost. And then also 2) there's no long term maintenance. When you actually go through and update this stuff, update the servers to like newer editions, it can be kind of a pain to be honest, especially if you're a small shop.
That kind of goes down to our next bullet point here is that the good thing about running off the cloud, even though this is on your desktop, is that the software updates automatically. You don't have to do anything about the servers. TIBCO takes care of all that for yourself. But as far as like what's running on your desktop, effectively you get the latest and greatest of all the software additions. Now, that can be a good thing and a bad thing, depending on if you're working with another entity. So there is a bit of a spread on the version history here where you can kind of go back 2 or 3, what they call, long-term solutions. So always keep this in mind if you're opening up a Spotfire following and it says hey, you've got an issue with compatibility, you do not want to save that file if you have to send it out to somebody who's on an older version. This is really key for those consultants out there like myself. We're typically running on a cloud, which means we're getting the fancier stuff that a lot of the clients don't have.
Also, the good thing about running on the cloud version is all the data tools, the data functions, and the web player features are automatically installed. OK, what does that mean? Data functions TERR (TIBCO Enterprise Runtime for R), so you can automatically be running those scripts if you still have them or are still using them from the get go. You don't have to contact IT to get them turned on. The big one here is also Python. Python started rolling out in Spotfire 3 or 4 years ago and has gotten much more powerful. Those are already enabled. You don't have to contact anybody to turn those things on. The other big one here is the web player, which is really, really nice. If you create Spotfire projects in-house and want to share them with clients or even share something on your website or send somebody a link really quickly, you don't have to send them the full file. You can actually just send them a URL through the web player. It can be a little bit of a pain to set this up when you're on the platform version, but with the cloud version it's super easy. If you ever want to look at an example like what that web player looks like, you can actually go out to our website. So it's www.sabata.us/directory is our biggest one. You can see where all of our data is actually hosted on a Spotfire project on the web player. Now that one is hosted on a Spotfire platform version, however, it doesn't really matter whether it's on their servers or your own, it's going to look the same. But again, what I'm getting at here is that these things are turned on from the beginning. As well as the data tools, these are some of the analytics tools. You can do some simple like multi-linear regression, stuff like that. These, I wouldn't say are super popular or super widely used in oil and gas. However, they are there if you want to mess around them.
Now, as far as the cons for the cloud version, a couple of different things: #1) information links. Now I am not the hugest fan of information links personally, however, they are really handy if you're working with really large datasets like pressure or any sort of SCADA data. So if you do need those and you're trying to use the info links, more importantly like cache data, you have to be on the Spotfire Platform, you cannot run that through Cloud. And I think the reason is that TIBCO really just doesn't want their servers to be kind of buggered up with all that data coming through.
Which leads us to the next bullet point is that the computing power. Because the cloud version... When you're looking at Spotfire, there's really 3 different entities that are running. There's your desktop or laptop, whatever your local machine is. That's what's actually kind of your user interface for what you're doing. They are also running the Spotfire server, which is doing really a lot of the bulk of the work and the calculations. And there's also a Spotfire database, which is another server in the background that's actually kind of temporarily storing your data while you're working through stuff. Now, if you're on the cloud version, you obviously own the local machine. But as far as the Spotfire database and Spotfire server, that's out there on AWS and TIBCO is handling those. Which is good, because you don't have to maintain those and stand them up. It's not very good though, because you're limited by their computing power. So if you're like me and are working with millions or tens of millions of data rows in a table, you can very easily overwhelm these systems and it'll slow you down, OK. So if you start getting to that point, you really need to move towards a platform version, OK.
The other con with the cloud version is permissions. So this is another reason why we normally stick to this with small teams. If you're in a small team, pretty much everybody's looking at the same files, right. You don't really have to set permissions on say, you don't want the production team messing around with the stuff in the geoscience team, right. For the most part, everyone's kind of staying in their lane and you don't really have those issues there. When you're working on bigger and bigger teams, especially with like information links, it's really hard to protect those because you can't set up custom folders and control the read vs. write permissions, in the cloud version, OK. So if you're running the cloud version, think of it as like a network drive. It's kind of free for all. Anybody can move around files, they can edit files, they can delete files, right. You really need to keep that in mind, OK.
Which leads us to the next one, which is the platform version. This is by far the most common solution for oil and gas companies. So just a reminder, this is where the Spotfire server and the Spotfire database portion are running on your own servers. Now, they used to be called on-prem, which is short for on-premise. The reason for that is more often than not, people running this off their own servers either in-house or at some data center that they've got at the airport or something like that. Now, you do not have to run that on-prem, you can run it out in the cloud. That's actually what we do here at Sabata. It works pretty well. It's just a little bit more set up there.
So the pros of the on-prem is 1) it's better for large teams. And the reason it's better for large teams is going back to those permissions, right. I can really, really separate out with like very exact precision who can see what, who can write what, who can edit stuff, who can delete stuff, and so on and so forth, right. So for IT, it's really where they have all the control and where they really like to work. Not always the best for the rest of us.
The good thing about the platform too, pretty much all the features that you see in Spotfire, you can turn on and off individually or IT can. Now in the cloud version, by default, literally everything is turned on. Which means if you know what you're looking for, there's a lot of stuff that you can pull out. If you don't know what you're looking for, it's really easy to kind of go down the rabbit hole like Alice and end up in a place you didn't want to be. So with the platform version, IT can turn on and off stuff explicitly, right. This really gets better... To me, honestly, the best reason that you do this is actually for the web player. Because stuff like data downloads, PowerPoint downloads, if people can save the file, the DXP files, stuff like that, those permissions are way more important when you're client facing. And if you're like me, you're trying to make sure that client information or our derivative information does not get out into the real world where people can see stuff through a web browser, but they cannot download it or have limited functionality or limited kind of ability to play with it, OK.
And as I mentioned a little bit before this, the big thing, the big pro for Spotfire Platform is the information links. So information links think of these, they are pre-built kind of SQL queries. It's not really exactly the same. It's kind of a middleman or saying, hey, I've got this database out here, I've got the Spotfire project here, I want my information links to essentially pull from one thing, clean it up, limit it, whatever like that and I'm going to pass along the line, OK. More often than not, frankly, it's just a middleman that kind of slows things up. The one exception to this is actually when you're working with like SCADA data or bottomhole pressure data for you reservoir engineers out there, OK. Typically, you're getting anywhere from 1-second data to 1-minute data to 50-minute data, blah, blah, blah, right. If you've got 5 wells, it's not a big deal. If you've got 100s or 1,000s of wells, this becomes a very big issue very quickly, OK. A very good example of this is like bottomhole pressure databases. I'm a reservoir engineer by background. I work with a lot of bottomhole pressure when I go through and do like well interference tests. If me and every other reservoir engineer is opening up the bottomhole pressure database every morning, every single time one of us hits that, not only are we hitting the Spotfire server saying hey, I want you to pull 10 million rows of data. I'm also hitting the server that you've got on the IT side, too. I guarantee your IT will not be happy if there's 10 of you all trying to pull 10 million data rows every 30 minutes, OK. This is where the info links, essentially, you can set these things up, recast the data. So whoever is the unlucky one or whoever's coming first thing in the morning, they open it up and it might take 10 or 15 minutes to pull this data in. But then after that, everybody else has got 30 seconds. The reason is that essentially those info links, they pull from the database, they cache it there for a set amount of time, typically 10 hours is what I usually set mine as. And then everybody after that is just kind of hitting that middleman. And then after 10 hours, it cleans it out and you start all over again. These are super, super handy, OK. I know that's a lot of detail there but I assure you, for those y'all in the big shops, you're familiar with the info links probably already. And if you have not been using the data caching, do it. It is absolutely worth your time. It will save you an enormous amount of loading time.
Now, for the cons on the Spotfire Platform, we really kind of cover this already in the cloud version. #1) you have to maintain those in-house servers. If you're like me, I'm an engineer and a geologist. I'm not an IT professional. I only deal with servers when I absolutely have to, right. And every time you have to deal with them, you're going to get slowed down, mainly because you have to reach out to somebody else, right, whether it's an IT person in-house or outside, right. So the good thing about the platform is that you can get some really souped up servers, right. You don't necessarily need that. We'll always tell people you don't need a Maserati to go to the grocery store. I mean, I guess if you really want to, you can. That being said, that really is going to control your cost there too, right. You want to get a souped up server? You can do some souped up stuff, but it's going to cost you souped up dollars too, OK. So that's just one more kind of cost up front and also an ongoing cost, i.e. the maintenance or the cloud cost to maintain those.
And then the big one here is the software updates. If you've talked to anybody that's worked with TIBCO over the years, especially in the IT side, they can tell you how updating the software can definitely be a pain. And there's always this real, I don't know if it's necessarily a risk, but there's always this real scare about when we update is it going to break things? Because the cloud version is out there, TIBCO is constantly sending out new versions, really like once a month. And you get a new long term solution. So this is a solution that's going to be maintained about every 3 - 4 months, OK. Why does that matter? Well, each Spotfire version can only really go back 2 or 3 long term solutions. So think about this as about every year. So for every year that you wait to update that stuff, you're essentially getting behind to where you can interact with people outside of your organization, OK. So if you rarely interact with people outside the organization, then it doesn't matter. You can do whatever you want to. I've worked with a lot of companies have been working off the old Spotfire versions for 5 or 6 years, which is an eon as far as software. However, if it works, it works, right. It doesn't really matter. However, updating that stuff can be a pain. So just keep that in mind, OK. And again, it is not always easy and it is most definitely not intuitive.

05. Acquiring an Academic License

OK, so for the academics or the students, researchers, whatever in the room too. I guess, if you're really of a non-profit. TIBCO does have academic licenses, although I will say, they do a very good job of hiding them, OK. So, they used to have a URL out there like specifically for academic licenses. It seems that they've actually gotten rid of that now. So if you're looking for one, just go out here. I'll give you a little URL here, which is a little screenshot for the sales and consulting. Tell them that you just want to get that academic license. The way that this used to work is that when you got the academic license, it was good for one year and they would lock you on to that Spotfire version. I'm not so sure if that is still the case. However, if you need one, it is available. They just don't mention it explicitly.
OK, and that's all there is for acquiring a license. Like I said, although that it looks like there's a lot more options, frankly, there's only a couple of different options that are really use and hopefully you find that information helpful.
https://sabata.us/https://www.tibco.com/products/tibco-spotfire/editions-pricing https://www.tibco.com/contact-us