Enterprise Geodatabase Maintenance

I have several enterprise level geodatabases (GDB) to manage. Instead of setting up maintenance scripts for each project GDB, I decided to write one script to manage them all. (Get the movie reference?)

This script requires two directories to be setup 1) OwnerFileDir and 2) AdminFileDir. The first will store several SDE owner connection files; one per database. Owner is usually "dbo" or "sde". The second will store several Esri object owner connection files; one per database. The Esri object owner is the user that owns each object's schema e.g. <DatabaseName>.<Schema>.<PolygonFeatureClassName>.
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
#------------------------------------------------------------------------------------------------
# Name: Enterprise GDB Maintenance
# Purpose: This script will compress a list of enterprise GDBs, update statistics, and  rebuild table indexes.
# Author:   Nicole Ceranek 
# Date Created: 12/28/2015
# Last Modified:    01/05/2016
# ArcGIS Version    10.1
# Python Version:   2.7
#------------------------------------------------------------------------------------------------
## Import modules
import arceditor
import os, arcpy, datetime, sys,time, os.path, logging, fnmatch, traceback
from arcpy import env
from datetime import date, timedelta, datetime
from time import ctime
from os import listdir
from os.path import isfile, join

## Set global variables
datetime120 = time.strftime("%Y%m%d_%H%M%S")
time108 = time.strftime("%H:%M:%S")
arcpy.env.overwriteOutput = True
# Owner directory
OwnerFileDir = "<enter your file path here>"
AdminFileDir = "<enter your file path here>"
# Admin directory
LogFile = open("<enter your file path here>/EGDBmaintenance_"+datetime120+".csv","w")
## Begin Script
ownerDB = os.listdir(OwnerFileDir)
for DB in ownerDB:
    try:
        arcpy.env.workspace = OwnerFileDir
        arcpy.AddMessage("Disconnecting users from "+DB+"\n")
        LogFile.write("Disconnecting users from,"+DB+"\n")
        arcpy.AcceptConnections(DB, False)
        arcpy.DisconnectUser(DB, "ALL")
        arcpy.AddMessage("Compressing "+DB+"\n")
        LogFile.write("Compressing,"+DB+"\n")
        arcpy.Compress_management(DB)
        arcpy.AddMessage("Allowing connections to "+DB+"\n")
        LogFile.write("Allowing connections to,"+DB+"\n")
        arcpy.AcceptConnections(DB, True)
    except arcpy.ExecuteError:
        msgs = arcpy.GetMessages(2)
        arcpy.AddError(msgs)
        arcpy.AddMessage(msgs)
        LogFile.write(msgs)
        ErrorCount=ErrorCount+1
    except PythonError:
        tb = sys.exc_info()[2]
        tbinfo = traceback.format_tb(tb)[0]
        pymsg = "PYTHON ERRORS:\nTraceback info:\n" + tbinfo + "\nError Info:\n" + str(sys.exc_info()[1])
        msgs = arcpy.GetMessages(2) + "\n"
        arcpy.AddError(pymsg)
        arcpy.AddError(msgs)
        arcpy.AddMessage(pymsg)
        LogFile.write(pymsg)
        arcpy.AddMessage(msgs)
        LogFile.write(msgs)
        ErrorCount=ErrorCount+1
    except:
        if (arcpy.DisconnectUser == "true"):
            arcpy.AcceptConnections(DB, True)
    finally:
        pass
# Rebuild indexes and analyze the states and states_lineages system tables for each EGDB connection file found in the admin directory
adminDB = os.listdir(AdminFileDir)
for DB in adminDB:
    try:
        arcpy.env.workspace = AdminFileDir+DB
        userName = arcpy.Describe(arcpy.env.workspace).connectionProperties.user
        oDataList = arcpy.ListTables('*.' + userName + '.*') + arcpy.ListFeatureClasses('*.' + userName + '.*') + arcpy.ListRasters('*.' + userName + '.*')
        for dataset in arcpy.ListDatasets('*.' + userName + '.*'):
            oDataList += arcpy.ListFeatureClasses(feature_dataset=dataset)
        LogFile.write("Tables owned by "+userName+":,"+str(oDataList)+",\n")
        arcpy.AddMessage("Rebuilding indexes for "+DB+"\n")
        LogFile.write("Rebuilding indexes for,"+DB+"\n")
        arcpy.RebuildIndexes_management(arcpy.env.workspace, "NO_SYSTEM", oDataList, "ALL")
        arcpy.AddMessage("Analyzing data for "+DB+"\n")
        LogFile.write("Analyzing data for,"+DB+"\n")
        arcpy.AnalyzeDatasets_management(arcpy.env.workspace, "NO_SYSTEM", oDataList, "ANALYZE_BASE", "ANALYZE_DELTA", "ANALYZE_ARCHIVE")
    except arcpy.ExecuteError:
        msgs = arcpy.GetMessages(2)
        arcpy.AddError(msgs)
        arcpy.AddMessage(msgs)
        LogFile.write(msgs)
        ErrorCount=ErrorCount+1
    except:
        tb = sys.exc_info()[2]
        tbinfo = traceback.format_tb(tb)[0]
        pymsg = "PYTHON ERRORS:\nTraceback info:\n" + tbinfo + "\nError Info:\n" + str(sys.exc_info()[1])
        msgs = arcpy.GetMessages(2) + "\n"
        arcpy.AddError(pymsg)
        arcpy.AddError(msgs)
        arcpy.AddMessage(pymsg)
        LogFile.write(pymsg)
        arcpy.AddMessage(msgs)
        LogFile.write(msgs)
        ErrorCount=ErrorCount+1
    finally:
        pass

Comments

  1. amadbearTue Feb 23, 09:36:00 AM

    F!@#ing awesome! didn't know you were keeping up with this. You should've added a free image referencing lord of the rings with your statement near the beginning "...one script to manage them all". Me likey.

    ReplyDelete
    Replies
    1. UnknownWed Feb 24, 08:17:00 PM

      I most certainly missed that opportunity! Glad you liked the post though. Thank you!

      Delete

Post a Comment

Popular posts from this blog

Updating GDB_Items XML Definitions

First, run a query to identify the feature class "PhysicalName" or UID from GDB_ITEMS. Note: The physical name appears as DBNAME.SCHEMA.FCNAME. Use one of the two for the WHERE clause in the other queries. First Query USE [DBName] SELECT ITEMS.ObjectID ,ITEMS.UUID ,ITEMTYPES.Name as ItemType ,ITEMS.[PhysicalName] ,ITEMS.[Definition] FROM [sde].[GDB_ITEMS] as ITEMS INNER JOIN [sde].[GDB_ITEMTYPES] ITEMTYPES ON [ITEMS].[ Type ] = [ITEMTYPES].[UUID] WHERE [PhysicalName] like '%FC Name%' In the results, click the XML link under the Definition field. You should see something similar to the XML below. Notice the highlighted line of code. This is the "Remarks" field. The node "IsNullable" is set as "false". I want the Remarks field to allow NULL values and would like to update it from "false" to "true". Note: The base and delta tables should be updated as well. XML Definition View <DEFeatu
Read more

Animation with ArcGIS Pro

Image
I collected data from Harris County Flood Control District and USGS for rainfall and stream heights.  I requested data between August 25th and September 5th, 2017 which was the time frame Hurricane Harvey rolled through the area. This animation shows the rainfall accumulation and stream height swell. The streams turn red once the height of the stream went above the top of bank. Here is the full video.
Read more

Add Field to a SQL Geodatabase using T-SQL

Image
Sometimes a new field will need to be added quickly. The usual process of testing changes and pushing said changes to production is not very time sensitive. Here is a method I use when a new field is needed ASAP. In this case the feature classes have been registered in the geodatabase and archiving has been enabled. This means there is an SDE base table and several SDE delta tables. The information needed to proceed is listed below. This will require DBO level access and SQL Server Management Studio or other suitable RDBMS interface like Toad. Database Name = DemoData New Field Name = NewField New Field Type (Length) = string(50) / nvarchar(50) SDE Field Type Code =   Unknown New Field Name = NewField Feature Class Name = Marker Base Table Name = Marker Base Table ID = Unknown Delta Table (Adds) = Unknown Delta Table (Archive) = Unknown First thing to do is find the unknown values. The SDE Field Type Code via Esri tells us that the SDE Field Type Code for String i
Read more