» Fig Leaf Software Home

We've Got You Covered.

Wednesday, September 10, 2014

Website Survey and a Chance to Win an iPad Mini!

In an effort to improve our website, we invite you to participate in a short survey. You will also have the opportunity to enter a raffle for an iPad Mini! Your input is very important to us and will be kept strictly confidential. 

The deadline for the survey is Wednesday, September 17, 2014.

For any questions about the survey or if you have trouble accessing the link, please email marketing@figleaf.com 

Complete the survey here: Fig Leaf Software Website Survey

We appreciate your input! 

Wednesday, August 13, 2014

Update your Skillz with New Node.JS Courseware!

Fig Leaf Software is proud to announce that Steve Drucker just finished authoring our new 1-day Node.JS Fundamentals course – weighing in at 110 pages with 9 hands-on exercises! This class will get you up-to-speed quickly on creating Node apps using the Express framework, dynamically constructing html output with Jade templates, and interacting with popular databases (MySQL, MongoDB, and CouchDB). It also covers socket I/O, File system access, and other exciting topics!

Check out the course outline and register today at: http://training.figleaf.com/courses/nodejs100.cfm

For information about licensing this course or for a sample to evaluate, please contact Steve directly at sdrucker@figleaf.com!

Friday, August 1, 2014

Comparison of Google Search Appliance 7.2 and 7.0 Admin Consoles

The latest major version of Google Search Appliance software, GSA 7.2, was released on 12 February 2014, and introduced a ton of new features. One of those features was a significantly redesigned admin console, with restructured navigation menus. Prior to 7.2, the admin console hadn't changed in many years, so when I started working with 7.2 I had a bit of trouble remembering where to find specific menu items in the new admin console. I've heard GSA customers and students having the same problems finding things. In fact, one person I talked to actually backed out of a system upgrade because he couldn't find items in the new console! (The ability to back out of a system upgrade is itself a new 7.2 feature, by the way.)

So that's the motivation behind this blog post. Within the 7.2 release notes, there's actually a specific list that maps old menu item locations to new ones, but I thought that a little visual representation might help. This post will help you navigate from GSA 7.0 to GSA 7.2.

Basically, the new admin console has mostly the same functionality as the old one. There are a few completely new menu items, but pretty much everything in the old menu is also in the new one. So, this is largely a matter of mapping specific menu items into new menu categories in the new admin console.

GSA Admin Console home screens - 7.0 on the left, 7.2 on the right
GSA Admin Console home screens - 7.0 on the left, 7.2 on the right.
With a little bit of explanation, I think it's pretty easy to understand the new organization. On GSA 7.0, we have Crawl and IndexServing, and Status and ReportsCrawl and Index covers any setting that controls the acquisition and storage of content. Serving covers any setting that controls search interactions. Status and Reports contains Crawl Status, the list of everything in the index (Crawl Diagnostics), all logs, and anything that can generate reports or other output for you to examine.

On GSA 7.2, we now have Content SourcesIndexSearch and ReportsContent Sources only covers the acquisition of content. Index only covers the storage of that content. Search covers any setting that controls search interactions at serve time - this is slightly different from Serving on GSA 7.0, but most menu items that were under Serving are now under SearchReports only has a few items now: Serving Logs, Search Reports and Search Logs. Most of the items previously under Status and Reports have been moved to Content Sources (Crawl Status, Crawl Queue, some of Real-time Diagnostics), Index (Content Statistics, Export URLs, and Index Diagnostics - formerly known as Crawl Diagnostics), Search (the rest of Real-time Diagnostics) and even Administration (Event Log).

Mapping GSA 7.0 menu items to GSA 7.2 menus

In the following series of screenshots, GSA 7.0 menu items are mapped to the corresponding GSA 7.2 menu items. The items that don't have a match within the currently displayed menus are highlighted in yellow. These items do have a match under another menu. The items that don't have a match at all in the other version are highlighted in green. There's only one of these, and it's a new feature in GSA 7.2: Trusted Applications.

Many GSA 7.0 Crawl and Index menu items are found within Content Sources on GSA 7.2.

GSA 7.0 Crawl and Index - GSA 7.2 Content Sources
GSA 7.0 Crawl and Index - GSA 7.2 Content Sources
The remaining GSA 7.0 Crawl and Index menu items can all be found under the Index menu on GSA 7.2.

GSA 7.0 Crawl and Index - GSA 7.2 Index
GSA 7.0 Crawl and Index - GSA 7.2 Index
All of the GSA 7.0 Serving menu items are found in GSA 7.2 Search except for Alerts and OneBox Modules. Trusted Applications in 7.2, highlighted in green below, is a new feature.

GSA 7.0 Serving - GSA 7.2 Search
GSA 7.0 Serving - GSA 7.2 Search
OneBox Modules are now found under Content Sources in 7.2.

GSA 7.0 Serving - GSA 7.2 Content Sources
GSA 7.0 Serving - GSA 7.2 Content Sources
Alerts are now found under Index in 7.2.

GSA 7.0 Serving - GSA 7.2 Index
GSA 7.0 Serving - GSA 7.2 Index
The menu items in GSA 7.0 Status and Reports, as described above, have been distributed to several menus in 7.2. Crawl Status, some parts of Real-time Diagnostics, and Crawl Queue are now under Content Sources.

GSA 7.0 Status and Reports - GSA 7.2 Content Sources
GSA 7.0 Status and Reports - GSA 7.2 Content Sources
Crawl Diagnostics, Content Statistics and Export URLs are now under Index in 7.2. Crawl Diagnostics is now known as Index Diagnostics.

GSA 7.0 Status and Reports - GSA 7.2 Index
GSA 7.0 Status and Reports - GSA 7.2 Index
Search Status and the remainder of Real-time Diagnostics are now under Search in GSA 7.2.

GSA 7.0 Status and Reports - GSA 7.2 Search
System Status and Event Log are now under Administration in GSA 7.2.

GSA 7.0 Status and Reports - GSA 7.2 Administration
GSA 7.0 Status and Reports - GSA 7.2 Administration
Serving Logs, Search Reports and Search Logs are under Reports in GSA 7.2.

GSA 7.0 Status and Reports - GSA 7.2 Reports
GSA 7.0 Status and Reports - GSA 7.2 Reports
Connector Managers and Connectors no longer have their own menu. In GSA 7.2, they're under Content Sources.

GSA 7.0 Connector Administration - GSA 7.2 Content Sources
Likewise, the Social Connect menu in GSA 7.0 is gone. User Results and Expert Search are now both under Search in GSA 7.2.

GSA 7.0 Social Connect - GSA 7.2 Search
GSA 7.0 Social Connect - GSA 7.2 Search
The Cloud Connect menu in GSA 7.0 has also been removed, and the single entry for Google Apps has been moved to Content Sources in GSA 7.2.

GSA 7.0 Cloud Connect - GSA 7.2 Content Sources
GSA 7.0 Cloud Connect - GSA 7.2 Content Sources
The GSA Unification and GSA^n menus haven't changed significantly across versions, although the design of some of the menu screens has changed a little bit. Still, everything's in the same place for these menus.

The Reset Index menu under Administration in GSA 7.0 is now under Index in GSA 7.2.

GSA 7.0 Administration - GSA 7.2 Index
GSA 7.0 Administration - GSA 7.2 Index

Menu behavior

The menus in GSA 7.2 behave a little differently. They're stored on the client, so they can be displayed as you click on them without a server-side page refresh. Clicking on a menu to expand it doesn't select the first menu item as it did in GSA 7.0, either. For example, in GSA 7.0, when you clicked on Crawl and Index, it would immediately show you the Start URLs page. In GSA 7.2, clicking on Content Sources just expands the menu, and you have to explicitly click on Start and Block URLs to view that. If you haven't actually clicked on a menu item in a while, your session may have expired. You can continue clicking on menus after your session has expired, but once you click on a menu item you'll be prompted to log in.

Another minor difference is that some menus have nested menus. For example, in the screenshot below, Content Sources and its submenus are highlighted in blue. Clicking on these submenus simply expands or collapses them, and it's pretty easy to inadvertently collapse one of these by accident.

GSA 7.2 Content Sources and submenus
GSA 7.2 Content Sources and submenus

Start and Block URLs page

The Start URLs page on GSA 7.0 was pretty simple: three HTML textareas for you to fill in. By default, in GSA 7.2 the Start and Block URLs page is quite a bit more complicated, in an attempt to make it easier to use. Each line item in each of the three sections (Start URLs, Follow Patterns, Do Not Follow Patterns) is a separate entry, and clicking on it opens a textarea containing just that item. 

GSA 7.2 - editing a single Start URL
GSA 7.2 - editing a single Start URL
Each line also has an Actions button and a Trash button. The Actions button lets you use that URL directly in Crawler Access, Index Diagnostics or a recrawl request!

GSA 7.2 - Start URL Actions options
GSA 7.2 - Start URL Actions options
However, you might find it's a lot harder to just type or edit URLs and patterns in this interface. In that case, click the Batch Edit radio button at the top right and you get the 7.0-style interface back. You can switch back and forth any time.

Conclusion

While the two admin consoles are pretty similar, enough things have been moved around and changed that you might get confused! This quick tour is intended to prevent that confusion, but if you have any questions, please feel free to send them to google@figleaf.com and we'll be happy to respond!

[Note: cross-posted on Dave Watts' personal blog]

Wednesday, July 30, 2014

Top 8 Things to Know About Private Training

Over the years, Fig Leaf Software has trained over 35,000 students in different courses from Google, Adobe, Acquia Drupal, Sencha and many others. Many people may not be aware, but Fig Leaf Software can deliver all of it’s training classes as private events! We have also travelled all over the world to train, and we can even deliver live-online classes in any time zone. Classes can be held at the client’s location, our location, or a neutral location that all students can attend. Here are some common questions we often receive about private training:

Is private training right for our organization?

If you have 4-5 students interested in the same class, private training is for you. Not only is private training cost effective, but one of the biggest advantages for setting up a private class is that you can customize the training to your needs. We routinely have clients who want a mixture of two classes, or simply want to remove or add a few units to the existing materials.

If I host the training at my location will I have to set up the computers?

Yes, however most classes only require about 5 minutes of setup per computer. Additionally, our training manager Jabari Bruce is available to assist with the setup. Worried about tough network restrictions? No worries, we have a few options when it comes to classroom setup, including using Amazon’s EC2 to host the class.

OK, we have a few students interested in training, where do we go from here?

Simple, just contact our training team at training@figleaf.com. If you are having any trouble we can help you decide which class(es) are right for you, and we can usually get you an estimate within 24 hours. We like to cap our classes at 12 students to keep the student to teacher ratio low, but we do have options for larger groups as well. Additionally, have all the students take a look at our class pre-requisites before the training starts. This can have a huge effect on how much the students get out of the class.

How long does it take to schedule a private class?

We can usually book a training class within 4-6 weeks, but if the instructor is available we may be able to get it scheduled sooner.
I have students in multiple locations all interested in training, can a private class still work?

Yes, we can teach all or a portion of the students online. This is live-online training, not a prerecorded video. We will setup an online environment, and provide all the remote students with a conference line so they can communicate with the instructor.

How does online training work?

The lecture component of the class will be handled through Adobe Connect and a phone conference bridge. For the best results you should have a dual-monitor system (or second computer). This helps replicate the classroom experience by watching the walkthrough exercises on one screen, and following along with the instructor on the other.

The access steps process:
  • Prior to the event you must run a connectivity test from their computer to make sure they can connect to our data centers. This ensures a smooth start when the training begins.
  • You will go to our hosted labs webpage to access the lab-environment remotely.
  • Using your assigned, unique access code you will be connected to a full-screen high performance remote desktop.

You can view more information about system requirements for online training at: http://training.figleaf.com/Locations/onlinefaq.cfm

Where do I find out if a class will run?

You can visit our training website: http://training.figleaf.com/index2.cfm. You can also contact us directly at 202-797-7711 x 107 or training@figleaf.com.

What should I estimate for travel costs?

For domestic classes you should estimate around $500 a day. You should estimate around $1,000 a day for international classes.

Schedule your private training course today by contacting our Training Coordinator, Jabari Bruce at training@figleaf.com or by phone at 202-797-7711 x107.

Friday, July 25, 2014

What's from China and has 8 legs?

It's one of those errors that could only happen with computers.

Apparently, if you run IIS,
and you run ColdFusion,
and maybe you run Commonspot, 
and a specific Chinese web crawler hits your website, then your website will crash.

For over a week, the client's website had been going down.

Just a few months ago they had been migrated to a new server and upgraded to the latest software.  Windows Server 2008 R2, ColdFusion 10, Commonspot 8 (The CMS), IIS 7.5.  

Things were fine when the server was first set up, but now, several times a day, ColdFusion would hit 100% processor usage and die, requiring a restart to get the site back online.

I traced that issue to a bad webservice call - a webservice the client website utilized heavily had become unreliable, and on those occasions when it didn't respond to requests, ColdFusion pages would hang indefinitely in the system, eventually sucking up all the resources and killing the website.

I fixed this by swapping some cfinvoke calls for straight cfhttp calls to the webservice - there's an issue with cfinvoke where at certain stages of the webservice invocation (like waiting for a response), ColdFusion will go into a holding pattern of just waiting, regardless of timeout settings.  Cfhttp calls worked every bit as well and timed out successfully, thus eliminating the problem.
But it didn't save the website.  That still died several times a day, but it died in a different way.  IIS would show pages with a 503 'Service Unavailable' error.  ColdFusion server monitoring and FusionReactor, useful in diagnosing the last issue, were showing no long-running pages this time.  The server resources never hit their capacity.  Everything appeared to be fine and dandy, even as the website died every few hours.
The webservice issue had masked something else.  Once the original killer was dead, another one reared its ugly head - the server either hadn't been running long enough for this one to be noticed, or whatever crashes it had caused in the interim were assumed to be the webservice issue.
Server event logs and Performance Monitoring of the application pools started to hint at the issue - I found errors like these in the event logs:

A process serving application pool 'DefaultAppPool' exceeded time limits during shut down. The process id was '6332'.  Event ID 5013

A worker process '6332' serving application pool 'DefaultAppPool' failed to stop a listener channel for protocol 'http' in the allotted time.  The data field contains the error number. Event ID 5138

A process serving application pool 'DefaultAppPool' suffered a fatal communication error with the Windows Process Activation Service. The process id was '3748'. The data field contains the error number. Event ID 5011
I found my go-to source of information - the internet - rather unhelpful in diagnosing these errors.  Suggestions included 'Find the error in your code' and 'Try replacing this .dll file with an older version of itself'.  After finding more promising info, I tried changing various application pool settings with no success.  

At a dead end, I looked for a way to try to get a better picture of what was going on inside the worker process threads.  You may need to install a server role (That role is IIS -> Heath and Diagnostics-> Request Monitor), as I did, to enable this.  Once it's there, from IIS you can select your server (not any of the websites), click on 'Worker Processes', and that will pop up a list of currently executing requests inside that worker process.
Inside a normal, healthy worker process you should see very little - a few requests executing with nice short times (those are in milliseconds), or maybe even nothing going on at the moment.

This is not what you want to see.

Ok, so that's a problem. 
My first thought was that there was something wrong with the pages, but no.  My own request for any of these pages would go through the application pool and CF, process, and a response was sent back to me - no problem.
My second thought was "Why are these all from similar IP addresses?"
A quick IP lookup revealed that the requests are coming from Guangzhou, China.  Odd traffic for an American site.  Armed with the IP address, I checked the IIS logs, where more of them showed up, helpfully self-identifying as 'Easouspider' requests.  There were several in the logs, and many of them completed successfully - anything that wasn't a .cfm page was fine, and there were successful .cfm page requests. 
Easou.com purports itself to be 'China's #1 mobile search engine', and they may well be - they're pretty big.  They're just crawling the web much like google and yahoo.  But for some reason, some requests from this particular crawler for .cfm pages for my client's server wouldn't go back out of the system.  They just sat in the application pools and nowhere else, gradually filling them up until something (usually errors) triggers application pool recycling.

Normal application pool recycling is pretty slick - IIS keeps the old pool running while it starts up the new one, and then does a handoff so the site doesn't drop, allowing all the requests in the old application pool to finish before fully transitioning to the new one.  That way your website doesn't appear to ever drop during a normal application pool transition.

That's when the truly diabolical thing was happening here - these hung requests are tough.  Persistent.  They do not want to let go.  IIS spends several minutes trying to wait for them to shut down so it can properly restart the application pool during a recycle - and that's the period when website would be dead, stuck in limbo with an old application pool that refused to die and unable to fully transition to the new one. 
Now, finding that the requests are stuck doesn't tell us exactly why it's happening.  The prevailing theory in the office is that requests from China are perhaps sending some odd character that IIS or CF doesn't like as part of the request, but even if we could narrow it down to that, what could we do about it?  I can't change how IIS or CF handles these requests.
So I blocked all IPs beginning with 183.60 from making requests to the site.  Better to cut off a part of China being able to access the site rather than allow the website to continue crashing.
It worked.  The site stopped crashing and ran smooth as butter.  The client was happy, I was happy, and all was well with the world.
A week later, another client called us in.  Their website was crashing mysteriously.

I checked the worker processes first this time, and there it was again.  Hung requests from IPs starting with 183.60.  The Chinese Spider was here too.  I had previously assumed this was an issue limited to the first client - despite having been upgraded recently, theirs was an older code base that had seen a lot of modification over the years, and nobody else had been crashing like they were until now.  This other client was a recent build, but apparently susceptible to the same issue.
After that, I checked every client we had - if two separate sites could go down like this, then others might be vulnerable as well.

I was right.  I found the Chinese Spider's requests stuck in two more ColdFusion / Commonspot / IIS websites that we manage (though several others got a clean bill of health, we nonetheless have blocked the IP on all our clients for now as a safety precaution).  In those cases it appears that the crawler hadn't been hitting the sites too hard, so the application pool would recycle naturally before dying from too many hung requests.
If you manage a ColdFusion / IIS site, I'd recommend taking a quick peek inside those worker threads - if it was present in four of ours, I'd wager we're not the only folks experiencing this strange issue, even if it isn't bringing down your website yet.

Written by: Benjamin Bently

About Us

Fig Leaf Software is an award-winning team of imaginative designers, innovative developers, experienced instructors, and insightful strategists.

For over 20 years, we’ve helped a diverse range of clients...

Read More

Contact Us

202-797-7711

Fig Leaf Software

1400 16th Street NW
Suite 450
Washington, DC 20036

info@figleaf.com