The list of data that needs to be profiled and clean are listed below. Click the link to see more detail on where we are on the process.
** priorities
| Data | Profiling | Cleaning | Transformation | Restructuring |
|---|---|---|---|---|
|
BlackKnight ** |
Completed (Arlington County) Completed (James City County) |
In progress (Arlington County) In progress (James City County) |
In progress (Arlington County) In progress (James City County) |
In progress (Arlington County) In progress (James City County) |
|
CoreLogic ** |
Completed (Arlington County) Completed (James City County) |
Completed (Arlington County)) Completed (James City County) |
Completed (Arlington County) Completed (James City County) |
Completed (Arlington County) Completed (James City County) |
| Completed | Completed | Completed | Completed | |
|
Completed (2010-2014) Completed (2009) |
Completed (2010-2014) Completed (2009) |
Completed (2010-2014) |
Completed (2010-2014) Completed (2009) |
|
| Completed | Completed | Completed | Completed | |
| Completed |
Completed |
Completed |
Completed | |
|
Arlington County Permits |
In progress | |||
| Completed | ||||
| Arlington County Crime | Completed | Completed | Completed | Completed |
| Completed | Completed |
Completed |
Not Needed | |
| Arlington County Economic Development | ||||
|
Completed |
Completed |
Completed |
Not Needed | |
|
Completed |
Completed |
Completed |
Completed | |
| James City County Permits | ||||
| HMDA |
Completed |
|||
|
USDA Tree Canopy |
Areas that Need to be Addressed
This section lists the areas of the profiling or cleaning process that requires further attention (e.g. definition clarification) in order to proceed.
| Addressed (Y/N)? | Issues |
|---|---|
| Y | General: Need to come up with common format of Addresses (Will base location on lat/lon) |
| Y | General: What year to base constant dollars? Should I also multiply by adjustment factor found in ACS data? 2013; no need to adjustment factor |
| Y | WMLS: Need to get parcel IDs for units. This will allow us then make unique identifiers for housing units and then restructure data where appropriate. Decided this step not needed (see profiling on what it would entail) |
| Y |
Real Estate Assessment: Figure out what provalLrsnId" means. Proval land record serial number: identification for all properties Factor |
| Y | Real Estate Assessment: Confirm what "Improvement.Value.Amount" means. An "improvement' usually means a home/house/structure of some type. |
| Y | Real Estate Assessment: Difference between "realEstatePropertyCode" and "masterRealEstatePropertyCode" realEstatePropertyCode is an identification for all properties and masterRealEstatePropertyCode" is specific for condo buildings only its really used for GIS display. |
| Y |
Real Estate Assessment: Confirm what "reasPropertyStatusCode” means (Levels: A, T) A: Active T: Inactive |
| Y | Real Estate Dwelling: What to do with 66 properties that appear more than once but have different information. No time stamp. Emailed both real estate and API people: All data in these tables is current. There are no duplicated Property-Dwelling properties |
| Y | Real Estate Interior: What to do with 34,365 (32% of data) properties that appear more than once but have different information. No time stamp. Emailed both real estate and API people: All data in these tables is current. There are no duplicated Property-Floor observations. |
| Y | Real Estate Property: some property list as having 1 unit count are also listed as being in high right apartments (NumberofUnits AND Apartment highrise/midrise is invalid) Made these unit counts NA |
| Y | Real Estate: multiple dwelling properties... how to handle when restructuring? Minimal effects on final estimates and doesn't effect all variables |
| Y | Real Estate: Is time-stamped data available? Yes, got the data from the department of Real Estate Assessment |
| JCC Parcel: What years are current and past assessment data from? Current year is 2015 | |
| Y | Real Estate: Check assessments that around low (~100) Removed empty land |
| Y | Real Estate: Why are some payments and levies missing? Set as 0 (non payment)? Mark those that have have some sort of reduction in price Included deferred/adjusted/relief category which explains most of the difference |
| Y |
Real Estate: Remove certain types of sales? See Description for which categories removed (those that are primarily non market value sales) |
| Y | Real Estate: Why do 58 have mismatched years? Multiple dwelling properties. Rename the Year Built Variable to Reflect this |
| Y | Real Estate: 2013 unit counts are off (some greater than 2,000), which makes the weighted values in the millions. Consistency check across the years shows 2013 errors. Emailed county. Switched systems which might have lead to error. Took previous year's value. |
| Y | Real Estate: Why are there still assessments that are low (~1000)? Parcels that later became inactive are still included so are some parking spots (that are not clearly marked) and new construction with low improvement value and 0 land value |
ACS Housing Tables and External Data
Arlington County
| ACS Table | BKFS | CoreLogic | AC Real Estate | MRIS MLS | |
|---|---|---|---|---|---|
| B25001 Housing Units | 5 Year | X | X | ||
| 1 Year | X | X | X | ||
| B25003 Occupancy Status | 5 Year | X | |||
| 1 Year | X | X | |||
| B25024 Units in Structure | 5 Year | X | X | ||
| 1 Year | X | X | X | ||
| B25034 Year Structure Built | 5 Year | X | X | ||
| 1 Year | X | X | X | X | |
| B25035 Median Year Structure Built | 5 Year | X | X | X | |
| 1 Year | X | X | X | X | |
| B25041 Bedrooms | 5 Year | X | X | X | |
| 1 Year | X | X | X | X | |
| B25075 Value | 5 Year | X | X | X | |
| 1 Year | X | X | X | X | |
| B25077 Median Value (Dollars) | 5 Year | X | X | X | |
| 1 Year | X | X | X | X | |
| B25102 Real Estate Taxes Paid | 5 Year | X | X | X | |
| 1 Year | X | X | X | X | |
James City County
| ACS Table | BKFS | CoreLogic | JCC Parcel | WMLS | |
|---|---|---|---|---|---|
| B25001 Housing Units | 5 Year | X | X | ||
| 1 Year | X | X | |||
| B25003 Occupancy Status | 5 Year | X | |||
| 1 Year | X | ||||
| B25024 Units in Structure | 5 Year | X | |||
| 1 Year | X | ||||
| B25034 Year Structure Built | 5 Year | X | X | X | |
| 1 Year | X | X | X | ||
| B25035 Median Year Structure Built | 5 Year | X | X | X | |
| 1 Year | X | X | X | ||
| B25041 Bedrooms | 5 Year | X | X | X | |
| 1 Year | X | X | X | ||
| B25075 Value | 5 Year | X | X | X | |
| 1 Year | X | X | X | ||
| B25077 Median Value (Dollars) | 5 Year | X | X | X | |
| 1 Year | X | X | X | ||
| B25102 real Estate Taxes Paid | 5 Year | X | |||
| 1 Year | X | ||||
NOTE: Highlighted tables are those of primary focus.
| ACS Table |
BlackKnight Variable Name (Field No.) |
CoreLogic | WMLS | MRIS-MLS | Location, Inc. | Arlington County Real Estate Assessments | Arlington County Permits | Arlington County Housing/ATRACK | James City County Parcel |
|---|---|---|---|---|---|---|---|---|---|
| B25001 Housing Units | X | X | |||||||
|
B25002 Occupancy Status |
Owner-Occupied (36) |
X | |||||||
|
B25024 Units In Structure |
No of Units (81) No of Buildings (79) |
X | X | X | X | ||||
|
B25034 Year Structure Built |
Year Built (78) Effective Year Built (149) |
X | X | X | X | X | X | ||
|
B25035 Median Year Structure Built |
X | X | X | X | X | X | X | ||
|
B25036 Tenure By Year Structure Built |
X | X | X | ||||||
|
B25037 Median Year Structure Built By Tenure |
X | X | |||||||
|
B25038 Tenure By Year Householder Moved Into Unit |
|||||||||
|
B25040 Housing Heating Fuel |
Heating (104) Heating Fuel Type (150) |
X | X | X | |||||
|
B25041 Bedrooms |
No of Bedrooms (83) Total # Rooms (82) Other Rooms (176) |
X | X | X | X | X | X | ||
|
B25042 Tenure By Bedrooms |
X | X | X | ||||||
|
B25047 Plumbing Facilities For All Housing Units |
# of Plumbing Fixtures (134) |
X | X | X | X | X | |||
|
B25048 Plumbing Facilities For Occupied Housing Units |
|||||||||
|
B25049 Tenure By Plumbing Facilities |
X | X | |||||||
|
B25051 Kitchen Facilities For All Housing Units |
|||||||||
|
B25063 Gross Rent |
|||||||||
|
B25064 Median Gross Rent (Dollars) |
X | ||||||||
|
B25066 Aggregate Gross Rent (Dollars) By Units In Structure |
|||||||||
|
B25068 Bedrooms By Gross Rent |
|||||||||
|
B25075 Value |
Total Assessed Value (39) = Assessed Land Value (37) + Assessed Improvement Value (38) Total Market Value (96) = Market Value: Land (94) + Market Value Improvement (95) Sale Price (52) Prior Sale Price (55) Tax Amount (44)
|
Calculated Assessments Market Appraisal |
Final Selling Price |
Original Listing Final Selling Price |
Assessments Selling Price |
Selling Price Assessments (no year) |
|||
|
B25077 Median Value (Dollars) |
X | X | X | X | X | X | |||
|
B25081 Mortgage Status |
Mortgage Lender Name? (153) | ||||||||
|
B25082 Aggregate Value (Dollars) By Mortgage Status |
|||||||||
|
B25096 Mortgage Status By Value |
|||||||||
|
B25102 Mortgage Status By Real Estate Taxes Paid |
|||||||||
|
B25103 Mortgage Status By Median Real Estate Taxes Paid (Dollars) |
|||||||||
|
B25107 Median Value By Year Structure |
X | X | X | X | X | X | |||
| B25082 Aggregate Value (Dollars) by Units in Structure | X | X | X | ||||||
|
B25111 Median Gross Rent (Dollars) By Year Structure Built |
X | ||||||||
|
B25114 Aggregate Gross Rent (Dollars) By Year Householder Moved Into Unit |
|||||||||
|
B25117 Tenure By House Heating Fuel |
X | X | |||||||
|
B25127 Tenure By Year Structure Built By Units In Structure |
X | X | X |